2024-09-12T07:08:48.121 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T07:08:48.137 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T07:08:48.226 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413 branch: wip-dis-testing description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} email: idryomov@gmail.com first_in_suite: false job_id: '7902413' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-dis-testing ceph: conf: client: rbd qos iops burst: 100 rbd qos iops limit: 50 rbd qos schedule tick min: 100 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd memory target: 1610612736 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: d61b08865d4d98738f45ff2e4806d7fb6d03eca1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d61b08865d4d98738f45ff2e4806d7fb6d03eca1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-dis-testing sha1: d61b08865d4d98738f45ff2e4806d7fb6d03eca1 owner: scheduled_dis@teuthology priority: 70 repo: https://github.com/ceph/ceph-ci roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 1680 sha1: d61b08865d4d98738f45ff2e4806d7fb6d03eca1 sleep_before_teardown: 0 subset: 42/100 suite: rbd suite_branch: wip-dis-testing suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d61b08865d4d98738f45ff2e4806d7fb6d03eca1/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci suite_sha1: d61b08865d4d98738f45ff2e4806d7fb6d03eca1 targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAcfZXe0VRHaEtk5S4R7vw7f/noGB1DFAB0c4pyLi6se0ro/S2JD050NXdiPBgqRPFJhugEW+viUsiyTy7zBj3I= smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPaE0wZSgmsbTYV6ikm6nO63TyJAIm/m0qwh5PQRHOzPocDz4O+fVtn0ERmZrGM1026FtAtrBKLqu8anICcxTBY= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-12_04:46:10 tube: smithi user: dis verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T07:08:48.227 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d61b08865d4d98738f45ff2e4806d7fb6d03eca1/qa; will attempt to use it 2024-09-12T07:08:48.227 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d61b08865d4d98738f45ff2e4806d7fb6d03eca1/qa/tasks 2024-09-12T07:08:48.227 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T07:08:48.229 INFO:teuthology.task.internal:Checking packages... 2024-09-12T07:08:48.250 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1' 2024-09-12T07:08:48.250 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T07:08:48.250 INFO:teuthology.packaging:ref: None 2024-09-12T07:08:48.250 INFO:teuthology.packaging:tag: None 2024-09-12T07:08:48.250 INFO:teuthology.packaging:branch: wip-dis-testing 2024-09-12T07:08:48.250 INFO:teuthology.packaging:sha1: d61b08865d4d98738f45ff2e4806d7fb6d03eca1 2024-09-12T07:08:48.250 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-dis-testing 2024-09-12T07:08:48.417 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4883.gd61b0886 2024-09-12T07:08:48.419 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T07:08:48.425 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T07:08:48.425 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T07:08:48.468 INFO:teuthology.task.internal:Saving configuration 2024-09-12T07:08:48.480 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T07:08:48.527 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T07:08:48.560 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 07:03:46.548340', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAcfZXe0VRHaEtk5S4R7vw7f/noGB1DFAB0c4pyLi6se0ro/S2JD050NXdiPBgqRPFJhugEW+viUsiyTy7zBj3I='} 2024-09-12T07:08:48.592 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 07:03:46.549931', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPaE0wZSgmsbTYV6ikm6nO63TyJAIm/m0qwh5PQRHOzPocDz4O+fVtn0ERmZrGM1026FtAtrBKLqu8anICcxTBY='} 2024-09-12T07:08:48.592 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T07:08:48.598 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-12T07:08:48.598 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-12T07:08:48.598 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T07:08:48.686 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9c5aa83ac0>, signals=[15]) 2024-09-12T07:08:48.686 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T07:08:48.693 INFO:teuthology.task.internal:Opening connections... 2024-09-12T07:08:48.693 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T07:08:48.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T07:08:48.778 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T07:08:48.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T07:08:48.861 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T07:08:48.867 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-09-12T07:08:48.886 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-09-12T07:08:48.886 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-09-12T07:08:48.944 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-09-12T07:08:48.944 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-09-12T07:08:48.944 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-09-12T07:08:48.944 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-09-12T07:08:48.944 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-09-12T07:08:48.944 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-09-12T07:08:48.944 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T07:08:48.944 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-09-12T07:08:48.944 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-09-12T07:08:48.945 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T07:08:48.945 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-09-12T07:08:48.945 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T07:08:48.945 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T07:08:48.945 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T07:08:48.945 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-09-12T07:08:48.988 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-09-12T07:08:49.006 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-09-12T07:08:49.006 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-09-12T07:08:49.063 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-09-12T07:08:49.063 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2024-09-12T07:08:49.063 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-09-12T07:08:49.064 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-09-12T07:08:49.064 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2024-09-12T07:08:49.064 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2024-09-12T07:08:49.064 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T07:08:49.064 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-09-12T07:08:49.064 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2024-09-12T07:08:49.064 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T07:08:49.064 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-09-12T07:08:49.064 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T07:08:49.064 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T07:08:49.064 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T07:08:49.065 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-09-12T07:08:49.096 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T07:08:49.104 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T07:08:49.135 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T07:08:49.135 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-09-12T07:08:49.138 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-09-12T07:08:49.154 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T07:08:49.161 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T07:08:49.161 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-09-12T07:08:49.195 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-09-12T07:08:49.224 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T07:08:49.296 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T07:08:49.310 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-12T07:08:49.311 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-12T07:08:49.311 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T07:08:49.311 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T07:08:49.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T07:08:49.328 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-12T07:08:49.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T07:08:49.329 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-09-12T07:08:49.386 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-503.el9.x86_64 2024-09-12T07:08:49.386 INFO:teuthology.task.kernel:Running kernel on smithi046: 5.14.0-503.el9.x86_64 2024-09-12T07:08:49.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum install -y kernel 2024-09-12T07:08:49.387 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-503.el9.x86_64 2024-09-12T07:08:49.388 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.14.0-503.el9.x86_64 2024-09-12T07:08:49.388 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y kernel 2024-09-12T07:08:52.067 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.2 MB 00:00 2024-09-12T07:08:52.186 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 7.8 MB/s | 8.2 MB 00:01 2024-09-12T07:08:53.937 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-12T07:08:54.685 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-12T07:09:00.093 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.5 MB 00:01 2024-09-12T07:09:00.412 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-12T07:09:01.627 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 19 kB 00:00 2024-09-12T07:09:02.779 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-12T07:09:02.873 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-12T07:09:04.706 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 13 MB/s | 23 MB 00:01 2024-09-12T07:09:08.504 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-09-12T07:09:10.264 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-09-12T07:09:10.569 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T07:09:10.569 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T07:09:10.655 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:09:10.663 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:09:10.663 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-09-12T07:09:10.663 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:09:10.663 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:09:10.663 INFO:teuthology.orchestra.run.smithi064.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T07:09:10.664 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-12T07:09:10.664 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T07:09:10.664 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T07:09:10.664 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T07:09:10.664 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:09:10.664 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:09:10.664 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:09:10.664 INFO:teuthology.orchestra.run.smithi064.stdout:Install 4 Packages 2024-09-12T07:09:10.664 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:09:10.665 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 79 M 2024-09-12T07:09:10.665 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 126 M 2024-09-12T07:09:10.665 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:09:11.005 INFO:teuthology.orchestra.run.smithi064.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 273 kB/s | 23 kB 00:00 2024-09-12T07:09:12.329 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T07:09:12.329 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T07:09:12.413 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:09:12.417 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:09:12.417 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-09-12T07:09:12.418 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:09:12.418 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:09:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T07:09:12.418 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-12T07:09:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T07:09:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T07:09:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T07:09:12.418 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:09:12.418 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:09:12.419 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:09:12.419 INFO:teuthology.orchestra.run.smithi046.stdout:Install 4 Packages 2024-09-12T07:09:12.419 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:09:12.419 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 79 M 2024-09-12T07:09:12.419 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 126 M 2024-09-12T07:09:12.419 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:09:12.624 INFO:teuthology.orchestra.run.smithi046.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 262 kB/s | 23 kB 00:00 2024-09-12T07:09:13.350 INFO:teuthology.orchestra.run.smithi046.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 39 MB/s | 29 MB 00:00 2024-09-12T07:09:13.797 INFO:teuthology.orchestra.run.smithi064.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.4 MB/s | 16 MB 00:02 2024-09-12T07:09:14.283 INFO:teuthology.orchestra.run.smithi046.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 20 MB/s | 35 MB 00:01 2024-09-12T07:09:16.572 INFO:teuthology.orchestra.run.smithi064.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.1 MB/s | 35 MB 00:05 2024-09-12T07:09:16.625 INFO:teuthology.orchestra.run.smithi046.stdout:(4/4): kernel-core-5.14.0-505.el9.x86_64.rpm 3.8 MB/s | 16 MB 00:04 2024-09-12T07:09:16.625 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:09:16.625 INFO:teuthology.orchestra.run.smithi046.stdout:Total 19 MB/s | 79 MB 00:04 2024-09-12T07:09:16.839 INFO:teuthology.orchestra.run.smithi064.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 4.9 MB/s | 29 MB 00:05 2024-09-12T07:09:16.839 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:09:16.839 INFO:teuthology.orchestra.run.smithi064.stdout:Total 13 MB/s | 79 MB 00:06 2024-09-12T07:09:17.332 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:09:17.547 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:09:17.724 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:09:17.725 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:09:17.908 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:09:17.909 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:09:18.711 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:09:18.711 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:09:18.916 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:09:18.916 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:09:21.300 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:09:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:09:21.967 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T07:09:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T07:09:22.215 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T07:09:22.297 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T07:09:24.926 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T07:09:25.060 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T07:09:25.082 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T07:09:25.309 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T07:09:30.184 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T07:09:30.211 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T07:09:30.437 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T07:09:30.491 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T07:09:35.354 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T07:09:35.633 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T07:10:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T07:10:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T07:10:14.521 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T07:10:14.563 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T07:10:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T07:10:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T07:10:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T07:10:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T07:10:14.921 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T07:10:14.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T07:10:14.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T07:10:14.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T07:10:15.504 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T07:10:15.504 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:10:15.504 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:10:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T07:10:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T07:10:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T07:10:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T07:10:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:10:15.505 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:10:15.770 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T07:10:15.771 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:10:15.771 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:10:15.771 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T07:10:15.771 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T07:10:15.771 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T07:10:15.771 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T07:10:15.772 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:10:15.772 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:10:15.824 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T07:10:15.908 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T07:10:15.908 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T07:10:15.908 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T07:10:15.909 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T07:10:15.909 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T07:10:15.909 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-12T07:10:15.909 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T07:10:15.909 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T07:10:15.968 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T07:10:15.968 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T07:10:15.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -qi grub2-tools 2024-09-12T07:10:15.999 INFO:teuthology.orchestra.run.smithi046.stdout:Name : grub2-tools 2024-09-12T07:10:16.000 INFO:teuthology.orchestra.run.smithi046.stdout:Epoch : 1 2024-09-12T07:10:16.000 INFO:teuthology.orchestra.run.smithi046.stdout:Version : 2.06 2024-09-12T07:10:16.000 INFO:teuthology.orchestra.run.smithi046.stdout:Release : 82.el9 2024-09-12T07:10:16.000 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: x86_64 2024-09-12T07:10:16.000 INFO:teuthology.orchestra.run.smithi046.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T07:10:16.000 INFO:teuthology.orchestra.run.smithi046.stdout:Group : Unspecified 2024-09-12T07:10:16.000 INFO:teuthology.orchestra.run.smithi046.stdout:Size : 8274670 2024-09-12T07:10:16.000 INFO:teuthology.orchestra.run.smithi046.stdout:License : GPLv3+ 2024-09-12T07:10:16.000 INFO:teuthology.orchestra.run.smithi046.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:Packager : builder@centos.org 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:Vendor : CentOS 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:Summary : Support tools for GRUB. 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:Description : 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:hardware devices. 2024-09-12T07:10:16.001 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:10:16.002 INFO:teuthology.orchestra.run.smithi046.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T07:10:16.002 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T07:10:16.002 INFO:teuthology.task.kernel:Updating grub on smithi046 to boot 5.14.0-505.el9.x86_64 2024-09-12T07:10:16.002 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T07:10:16.146 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T07:10:16.235 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T07:10:16.235 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T07:10:16.235 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T07:10:16.235 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T07:10:16.235 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T07:10:16.236 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-12T07:10:16.236 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T07:10:16.236 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T07:10:16.294 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T07:10:16.294 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T07:10:16.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2024-09-12T07:10:16.337 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2024-09-12T07:10:16.337 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2024-09-12T07:10:16.337 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.06 2024-09-12T07:10:16.337 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 82.el9 2024-09-12T07:10:16.337 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2024-09-12T07:10:16.337 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T07:10:16.337 INFO:teuthology.orchestra.run.smithi064.stdout:Group : Unspecified 2024-09-12T07:10:16.337 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 8274670 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : builder@centos.org 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T07:10:16.338 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T07:10:16.339 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T07:10:16.339 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2024-09-12T07:10:16.339 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:10:16.339 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T07:10:16.340 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T07:10:16.340 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 5.14.0-505.el9.x86_64 2024-09-12T07:10:16.340 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T07:10:16.591 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-09-12T07:10:16.935 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2024-09-12T07:10:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T07:10:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-09-12T07:10:18.030 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-09-12T07:10:18.045 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.eXD7TuvwpB 2024-09-12T07:10:18.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.eXD7TuvwpB 2024-09-12T07:10:18.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.eXD7TuvwpB 2024-09-12T07:10:18.257 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.eXD7TuvwpB is 6KB 2024-09-12T07:10:18.276 INFO:teuthology.orchestra.run.smithi064.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T07:10:18.305 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.eXD7TuvwpB 2024-09-12T07:10:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:done 2024-09-12T07:10:18.318 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-09-12T07:10:18.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T07:10:18.333 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.Iw1NZ5mmdq 2024-09-12T07:10:18.333 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Iw1NZ5mmdq 2024-09-12T07:10:18.389 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T07:10:18.389 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T07:10:18.389 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T07:10:18.389 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T07:10:18.390 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T07:10:18.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.Iw1NZ5mmdq 2024-09-12T07:10:18.545 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-09-12T07:10:18.555 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.Iw1NZ5mmdq is 6KB 2024-09-12T07:10:18.604 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.Iw1NZ5mmdq 2024-09-12T07:10:18.618 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T07:10:18.680 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T07:10:18.680 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T07:10:18.680 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T07:10:18.680 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T07:10:18.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T07:10:18.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2024-09-12T07:10:48.577 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-12T07:10:48.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T07:10:48.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T07:10:48.829 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-12T07:10:48.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-12T07:10:48.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T07:11:23.197 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-12T07:11:32.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T07:11:32.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T07:11:48.857 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T07:11:50.589 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-12T07:11:57.865 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-12T07:11:57.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T07:11:58.216 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-09-12T07:11:58.669 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-12T07:11:58.670 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-12T07:11:58.670 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-09-12T07:11:58.687 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-505.el9.x86_64 2024-09-12T07:11:58.688 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T07:11:58.688 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T07:11:58.688 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T07:11:59.688 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-12T07:11:59.689 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T07:11:59.798 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2024-09-12T07:11:59.830 DEBUG:teuthology.parallel:result is None 2024-09-12T07:12:02.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T07:12:02.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T07:12:02.911 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-12T07:12:03.329 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T07:12:03.329 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-12T07:12:03.329 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-12T07:12:03.348 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-505.el9.x86_64 2024-09-12T07:12:03.349 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T07:12:03.349 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T07:12:03.349 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T07:12:04.349 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-12T07:12:04.350 DEBUG:teuthology.orchestra.run.smithi046:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T07:12:04.476 INFO:teuthology.orchestra.run.smithi046.stdout:ttyS1 2024-09-12T07:12:04.517 DEBUG:teuthology.parallel:result is None 2024-09-12T07:12:04.517 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T07:12:04.524 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T07:12:04.524 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T07:12:04.526 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T07:12:04.544 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T07:12:04.554 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T07:12:04.587 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T07:12:04.587 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T07:12:04.590 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T07:12:04.613 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T07:12:04.619 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T07:12:04.619 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T07:12:04.648 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T07:12:04.676 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T07:12:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T07:12:04.687 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T07:12:04.689 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T07:12:04.690 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T07:12:04.794 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T07:12:04.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T07:12:04.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T07:12:04.823 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T07:12:04.839 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T07:12:04.840 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T07:12:04.864 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T07:12:04.879 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T07:12:04.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T07:12:05.003 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T07:12:05.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T07:12:05.126 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:12:05.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T07:12:05.191 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T07:12:05.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T07:12:05.278 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T07:12:05.382 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T07:12:05.416 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:12:05.416 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T07:12:05.482 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-09-12T07:12:05.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-09-12T07:12:05.541 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T07:12:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T07:12:05.934 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T07:12:06.065 INFO:teuthology.task.internal:Starting timer... 2024-09-12T07:12:06.065 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T07:12:06.144 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T07:12:06.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-09-12T07:12:06.205 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-09-12T07:12:06.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-09-12T07:12:06.325 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-09-12T07:12:06.329 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T07:12:06.329 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-09-12T07:12:06.358 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-09-12T07:12:06.358 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-09-12T07:12:06.421 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-09-12T07:12:06.421 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi064.front.sepia.ceph.com': 'permissive'} 2024-09-12T07:12:06.422 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-12T07:12:06.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T07:12:06.450 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-12T07:12:06.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T07:12:06.517 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T07:12:06.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-09-12T07:12:06.549 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-09-12T07:12:06.761 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T07:12:06.771 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T07:12:06.783 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T07:12:06.784 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi064.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T07:16:02.938 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi064.front.sepia.ceph.com')] 2024-09-12T07:16:02.939 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T07:16:02.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T07:16:03.041 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-12T07:16:03.122 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T07:16:03.123 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-12T07:16:03.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T07:16:03.211 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-09-12T07:16:03.289 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-12T07:16:03.290 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T07:16:03.301 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T07:16:03.302 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T07:16:03.302 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T07:16:03.305 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T07:16:03.305 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T07:16:03.337 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T07:16:03.354 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T07:16:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T07:16:03.377 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T07:16:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-09-12T07:16:03.394 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-09-12T07:16:03.403 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-09-12T07:16:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T07:16:03.414 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-09-12T07:16:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T07:16:03.430 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T07:16:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T07:16:03.483 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-09-12T07:16:03.486 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T07:16:03.486 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-09-12T07:16:03.486 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T07:16:03.486 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T07:16:03.487 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T07:16:03.487 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T07:16:03.508 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-09-12T07:16:03.512 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T07:16:03.512 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-09-12T07:16:03.513 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T07:16:03.513 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T07:16:03.513 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T07:16:03.513 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T07:16:03.513 INFO:teuthology.run_tasks:Running task install... 2024-09-12T07:16:03.522 DEBUG:teuthology.task.install:project ceph 2024-09-12T07:16:03.522 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}} 2024-09-12T07:16:03.522 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'} 2024-09-12T07:16:03.522 INFO:teuthology.task.install:Using flavor: default 2024-09-12T07:16:03.527 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T07:16:03.528 INFO:teuthology.task.install:extra packages: [] 2024-09-12T07:16:03.528 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1', 'tag': None, 'wait_for_package': False} 2024-09-12T07:16:03.528 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d61b08865d4d98738f45ff2e4806d7fb6d03eca1 2024-09-12T07:16:03.530 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1', 'tag': None, 'wait_for_package': False} 2024-09-12T07:16:03.530 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d61b08865d4d98738f45ff2e4806d7fb6d03eca1 2024-09-12T07:16:03.685 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/centos/9/flavors/default/ 2024-09-12T07:16:03.685 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4883.gd61b0886 2024-09-12T07:16:03.697 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/centos/9/flavors/default/ 2024-09-12T07:16:03.697 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4883.gd61b0886 2024-09-12T07:16:03.813 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T07:16:03.813 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:16:03.813 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T07:16:03.821 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T07:16:03.822 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:16:03.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T07:16:03.840 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-12T07:16:03.841 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T07:16:03.850 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-12T07:16:03.851 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T07:16:03.908 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T07:16:03.919 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T07:16:03.985 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T07:16:03.996 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T07:16:04.050 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-09-12T07:16:04.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-09-12T07:16:04.062 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-09-12T07:16:04.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-09-12T07:16:04.294 INFO:teuthology.orchestra.run.smithi064.stdout:45 files removed 2024-09-12T07:16:04.311 INFO:teuthology.orchestra.run.smithi046.stdout:45 files removed 2024-09-12T07:16:04.321 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw 2024-09-12T07:16:04.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw 2024-09-12T07:16:04.899 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-12T07:16:04.909 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 310 kB/s | 84 kB 00:00 2024-09-12T07:16:05.183 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-12T07:16:05.209 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-09-12T07:16:05.459 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-12T07:16:05.493 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T07:16:06.118 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-12T07:16:07.368 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.2 MB 00:01 2024-09-12T07:16:08.185 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-12T07:16:09.336 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-12T07:16:13.653 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-12T07:16:13.779 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-12T07:16:15.530 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-12T07:16:15.995 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-12T07:16:16.290 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-12T07:16:19.405 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 6.9 MB/s | 23 MB 00:03 2024-09-12T07:16:21.967 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-12T07:16:24.414 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:16:24.431 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:16:24.431 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-12T07:16:24.431 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:16:24.431 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 11 M 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 3.5 M 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 3.2 M 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 5.5 M 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 22 M 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 25 k 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T07:16:24.432 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 755 k 2024-09-12T07:16:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-12T07:16:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T07:16:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T07:16:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T07:16:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 508 k 2024-09-12T07:16:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T07:16:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 5.5 M 2024-09-12T07:16:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T07:16:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T07:16:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T07:16:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 45 k 2024-09-12T07:16:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 143 k 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 162 k 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 321 k 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 307 k 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 99 k 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:16:24.434 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:16:24.435 INFO:teuthology.orchestra.run.smithi046.stdout:Install 31 Packages 2024-09-12T07:16:24.435 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 2 Packages 2024-09-12T07:16:24.435 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:16:24.435 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 62 M 2024-09-12T07:16:24.435 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:16:25.049 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-09-12T07:16:25.352 INFO:teuthology.orchestra.run.smithi046.stdout:(1/33): ceph-base-19.3.0-4883.gd61b0886.el9.x86 11 MB/s | 5.5 MB 00:00 2024-09-12T07:16:25.419 INFO:teuthology.orchestra.run.smithi046.stdout:(2/33): ceph-selinux-19.3.0-4883.gd61b0886.el9. 378 kB/s | 25 kB 00:00 2024-09-12T07:16:25.561 INFO:teuthology.orchestra.run.smithi046.stdout:(3/33): ceph-radosgw-19.3.0-4883.gd61b0886.el9. 15 MB/s | 11 MB 00:00 2024-09-12T07:16:25.594 INFO:teuthology.orchestra.run.smithi046.stdout:(4/33): libcephfs2-19.3.0-4883.gd61b0886.el9.x8 4.2 MB/s | 755 kB 00:00 2024-09-12T07:16:25.628 INFO:teuthology.orchestra.run.smithi046.stdout:(5/33): libradosstriper1-19.3.0-4883.gd61b0886. 7.4 MB/s | 508 kB 00:00 2024-09-12T07:16:25.695 INFO:teuthology.orchestra.run.smithi046.stdout:(6/33): python3-ceph-argparse-19.3.0-4883.gd61b 675 kB/s | 45 kB 00:00 2024-09-12T07:16:25.779 INFO:teuthology.orchestra.run.smithi046.stdout:(7/33): python3-ceph-common-19.3.0-4883.gd61b08 1.7 MB/s | 143 kB 00:00 2024-09-12T07:16:25.862 INFO:teuthology.orchestra.run.smithi046.stdout:(8/33): librgw2-19.3.0-4883.gd61b0886.el9.x86_6 21 MB/s | 5.5 MB 00:00 2024-09-12T07:16:25.887 INFO:teuthology.orchestra.run.smithi046.stdout:(9/33): python3-cephfs-19.3.0-4883.gd61b0886.el 1.5 MB/s | 162 kB 00:00 2024-09-12T07:16:25.912 INFO:teuthology.orchestra.run.smithi046.stdout:(10/33): python3-rados-19.3.0-4883.gd61b0886.el 6.3 MB/s | 321 kB 00:00 2024-09-12T07:16:25.938 INFO:teuthology.orchestra.run.smithi046.stdout:(11/33): python3-rbd-19.3.0-4883.gd61b0886.el9. 6.0 MB/s | 307 kB 00:00 2024-09-12T07:16:25.963 INFO:teuthology.orchestra.run.smithi046.stdout:(12/33): python3-rgw-19.3.0-4883.gd61b0886.el9. 1.9 MB/s | 99 kB 00:00 2024-09-12T07:16:26.189 INFO:teuthology.orchestra.run.smithi046.stdout:(13/33): ceph-common-19.3.0-4883.gd61b0886.el9. 16 MB/s | 22 MB 00:01 2024-09-12T07:16:26.247 INFO:teuthology.orchestra.run.smithi046.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 346 kB/s | 107 kB 00:00 2024-09-12T07:16:26.297 INFO:teuthology.orchestra.run.smithi046.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-09-12T07:16:26.381 INFO:teuthology.orchestra.run.smithi046.stdout:(16/33): libnbd-1.20.2-2.el9.x86_64.rpm 408 kB/s | 171 kB 00:00 2024-09-12T07:16:26.414 INFO:teuthology.orchestra.run.smithi046.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 710 kB/s | 160 kB 00:00 2024-09-12T07:16:26.440 INFO:teuthology.orchestra.run.smithi046.stdout:(18/33): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-12T07:16:26.465 INFO:teuthology.orchestra.run.smithi046.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 839 kB/s | 42 kB 00:00 2024-09-12T07:16:26.499 INFO:teuthology.orchestra.run.smithi046.stdout:(20/33): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T07:16:26.524 INFO:teuthology.orchestra.run.smithi046.stdout:(21/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-12T07:16:26.558 INFO:teuthology.orchestra.run.smithi046.stdout:(22/33): qatlib-service-24.02.0-1.el9.x86_64.rp 388 kB/s | 36 kB 00:00 2024-09-12T07:16:26.583 INFO:teuthology.orchestra.run.smithi046.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 562 kB/s | 47 kB 00:00 2024-09-12T07:16:26.825 INFO:teuthology.orchestra.run.smithi046.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 107 kB/s | 26 kB 00:00 2024-09-12T07:16:26.917 INFO:teuthology.orchestra.run.smithi046.stdout:(25/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 783 kB/s | 308 kB 00:00 2024-09-12T07:16:26.951 INFO:teuthology.orchestra.run.smithi046.stdout:(26/33): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-09-12T07:16:26.993 INFO:teuthology.orchestra.run.smithi046.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 898 kB/s | 67 kB 00:00 2024-09-12T07:16:27.076 INFO:teuthology.orchestra.run.smithi046.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-12T07:16:27.243 INFO:teuthology.orchestra.run.smithi046.stdout:(29/33): libarrow-9.0.0-11.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-09-12T07:16:27.269 INFO:teuthology.orchestra.run.smithi046.stdout:(30/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.6 MB/s | 839 kB 00:00 2024-09-12T07:16:27.310 INFO:teuthology.orchestra.run.smithi046.stdout:(31/33): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-12T07:16:27.487 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:16:27.496 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:16:27.496 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-12T07:16:27.496 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 11 M 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 3.5 M 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 3.2 M 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 5.5 M 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 22 M 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 25 k 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T07:16:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 755 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 508 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 5.5 M 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 45 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 143 k 2024-09-12T07:16:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 162 k 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 321 k 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 307 k 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 99 k 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout:Install 31 Packages 2024-09-12T07:16:27.499 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 2 Packages 2024-09-12T07:16:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:16:27.500 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 62 M 2024-09-12T07:16:27.500 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:16:27.511 INFO:teuthology.orchestra.run.smithi046.stdout:(32/33): librados2-19.3.0-4883.gd61b0886.el9.x8 13 MB/s | 3.5 MB 00:00 2024-09-12T07:16:27.569 INFO:teuthology.orchestra.run.smithi046.stdout:(33/33): librbd1-19.3.0-4883.gd61b0886.el9.x86_ 11 MB/s | 3.2 MB 00:00 2024-09-12T07:16:27.575 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:16:27.575 INFO:teuthology.orchestra.run.smithi046.stdout:Total 20 MB/s | 62 MB 00:03 2024-09-12T07:16:27.698 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:16:27.739 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:16:27.739 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:16:28.450 INFO:teuthology.orchestra.run.smithi064.stdout:(1/33): ceph-base-19.3.0-4883.gd61b0886.el9.x86 7.0 MB/s | 5.5 MB 00:00 2024-09-12T07:16:28.453 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:16:28.453 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:16:28.492 INFO:teuthology.orchestra.run.smithi064.stdout:(2/33): ceph-selinux-19.3.0-4883.gd61b0886.el9. 606 kB/s | 25 kB 00:00 2024-09-12T07:16:28.626 INFO:teuthology.orchestra.run.smithi064.stdout:(3/33): ceph-radosgw-19.3.0-4883.gd61b0886.el9. 11 MB/s | 11 MB 00:00 2024-09-12T07:16:28.660 INFO:teuthology.orchestra.run.smithi064.stdout:(4/33): libcephfs2-19.3.0-4883.gd61b0886.el9.x8 4.4 MB/s | 755 kB 00:00 2024-09-12T07:16:28.693 INFO:teuthology.orchestra.run.smithi064.stdout:(5/33): libradosstriper1-19.3.0-4883.gd61b0886. 7.4 MB/s | 508 kB 00:00 2024-09-12T07:16:28.760 INFO:teuthology.orchestra.run.smithi064.stdout:(6/33): python3-ceph-argparse-19.3.0-4883.gd61b 677 kB/s | 45 kB 00:00 2024-09-12T07:16:28.827 INFO:teuthology.orchestra.run.smithi064.stdout:(7/33): python3-ceph-common-19.3.0-4883.gd61b08 2.1 MB/s | 143 kB 00:00 2024-09-12T07:16:28.894 INFO:teuthology.orchestra.run.smithi064.stdout:(8/33): python3-cephfs-19.3.0-4883.gd61b0886.el 2.4 MB/s | 162 kB 00:00 2024-09-12T07:16:29.111 INFO:teuthology.orchestra.run.smithi064.stdout:(9/33): ceph-common-19.3.0-4883.gd61b0886.el9.x 15 MB/s | 22 MB 00:01 2024-09-12T07:16:29.136 INFO:teuthology.orchestra.run.smithi064.stdout:(10/33): python3-rados-19.3.0-4883.gd61b0886.el 1.3 MB/s | 321 kB 00:00 2024-09-12T07:16:29.211 INFO:teuthology.orchestra.run.smithi064.stdout:(11/33): librgw2-19.3.0-4883.gd61b0886.el9.x86_ 10 MB/s | 5.5 MB 00:00 2024-09-12T07:16:29.237 INFO:teuthology.orchestra.run.smithi064.stdout:(12/33): python3-rgw-19.3.0-4883.gd61b0886.el9. 992 kB/s | 99 kB 00:00 2024-09-12T07:16:29.270 INFO:teuthology.orchestra.run.smithi064.stdout:(13/33): python3-rbd-19.3.0-4883.gd61b0886.el9. 1.9 MB/s | 307 kB 00:00 2024-09-12T07:16:29.412 INFO:teuthology.orchestra.run.smithi064.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 533 kB/s | 107 kB 00:00 2024-09-12T07:16:29.437 INFO:teuthology.orchestra.run.smithi064.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 850 kB/s | 171 kB 00:00 2024-09-12T07:16:29.463 INFO:teuthology.orchestra.run.smithi064.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 834 kB/s | 160 kB 00:00 2024-09-12T07:16:29.488 INFO:teuthology.orchestra.run.smithi064.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 599 kB/s | 45 kB 00:00 2024-09-12T07:16:29.555 INFO:teuthology.orchestra.run.smithi064.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-12T07:16:29.580 INFO:teuthology.orchestra.run.smithi064.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 460 kB/s | 42 kB 00:00 2024-09-12T07:16:29.641 INFO:teuthology.orchestra.run.smithi064.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-12T07:16:29.664 INFO:teuthology.orchestra.run.smithi064.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 429 kB/s | 36 kB 00:00 2024-09-12T07:16:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-09-12T07:16:29.723 INFO:teuthology.orchestra.run.smithi064.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 564 kB/s | 47 kB 00:00 2024-09-12T07:16:29.798 INFO:teuthology.orchestra.run.smithi064.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-09-12T07:16:29.849 INFO:teuthology.orchestra.run.smithi064.stdout:(25/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.6 MB/s | 308 kB 00:00 2024-09-12T07:16:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:(26/33): liboath-2.6.7-2.el9.x86_64.rpm 644 kB/s | 49 kB 00:00 2024-09-12T07:16:29.908 INFO:teuthology.orchestra.run.smithi064.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-12T07:16:29.975 INFO:teuthology.orchestra.run.smithi064.stdout:(28/33): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-12T07:16:29.981 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:16:30.000 INFO:teuthology.orchestra.run.smithi064.stdout:(29/33): re2-20211101-20.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-12T07:16:30.050 INFO:teuthology.orchestra.run.smithi064.stdout:(30/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-09-12T07:16:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-12T07:16:30.092 INFO:teuthology.orchestra.run.smithi064.stdout:(31/33): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-12T07:16:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-12T07:16:30.142 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-12T07:16:30.164 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-12T07:16:30.189 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-12T07:16:30.276 INFO:teuthology.orchestra.run.smithi064.stdout:(32/33): librados2-19.3.0-4883.gd61b0886.el9.x8 13 MB/s | 3.5 MB 00:00 2024-09-12T07:16:30.343 INFO:teuthology.orchestra.run.smithi064.stdout:(33/33): librbd1-19.3.0-4883.gd61b0886.el9.x86_ 11 MB/s | 3.2 MB 00:00 2024-09-12T07:16:30.345 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:16:30.345 INFO:teuthology.orchestra.run.smithi064.stdout:Total 22 MB/s | 62 MB 00:02 2024-09-12T07:16:30.393 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-12T07:16:30.455 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-12T07:16:30.467 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:16:30.508 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:16:30.508 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:16:30.872 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-12T07:16:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:19.3.0-4883.gd61b0886.el9.x86_64 7/35 2024-09-12T07:16:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:19.3.0-4883.gd61b0886.el9.x86_64 7/35 2024-09-12T07:16:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:19.3.0-4883.gd61b0886.el9.x86_64 8/35 2024-09-12T07:16:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:19.3.0-4883.gd61b0886.el9.x86_64 9/35 2024-09-12T07:16:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:19.3.0-4883.gd61b0886.el9.x86_64 9/35 2024-09-12T07:16:31.196 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:16:31.197 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:16:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-12T07:16:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:19.3.0-4883.gd61b0886.el9. 11/35 2024-09-12T07:16:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:19.3.0-4883.gd61b0886.el9.x86_64 12/35 2024-09-12T07:16:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:19.3.0-4883.gd61b0886.el9.x86_6 13/35 2024-09-12T07:16:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4883.gd61b0886.el9.x86_6 13/35 2024-09-12T07:16:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-12T07:16:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-12T07:16:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-12T07:16:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-12T07:16:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-12T07:16:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-12T07:16:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-12T07:16:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-12T07:16:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-12T07:16:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:19.3.0-4883.gd61b0886.el9.x86_64 23/35 2024-09-12T07:16:32.279 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:19.3.0-4883.gd61b0886.el9.x86_64 23/35 2024-09-12T07:16:32.308 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:19.3.0-4883.gd61b0886.el9.x86_64 24/35 2024-09-12T07:16:32.353 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-12T07:16:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:16:32.836 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-12T07:16:32.864 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-12T07:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-12T07:16:32.900 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-12T07:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-12T07:16:32.931 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:19.3.0-4883.gd61b0886.el9.x86_64 27/35 2024-09-12T07:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-12T07:16:32.986 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:19.3.0-4883.gd61b0886.el9.x86_64 27/35 2024-09-12T07:16:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:19.3.0-4883.gd61b0886.el9.x86_64 28/35 2024-09-12T07:16:33.116 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:19.3.0-4883.gd61b0886.el9.x8 29/35 2024-09-12T07:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-12T07:16:33.185 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-12T07:16:33.547 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-12T07:16:33.658 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:19.3.0-4883.gd61b0886.el9.x86_64 7/35 2024-09-12T07:16:33.752 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:19.3.0-4883.gd61b0886.el9.x86_64 7/35 2024-09-12T07:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:19.3.0-4883.gd61b0886.el9.x86_64 8/35 2024-09-12T07:16:33.884 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:19.3.0-4883.gd61b0886.el9.x86_64 9/35 2024-09-12T07:16:33.956 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:19.3.0-4883.gd61b0886.el9.x86_64 9/35 2024-09-12T07:16:33.993 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-12T07:16:34.037 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:19.3.0-4883.gd61b0886.el9. 11/35 2024-09-12T07:16:34.107 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:19.3.0-4883.gd61b0886.el9.x86_64 12/35 2024-09-12T07:16:34.136 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:19.3.0-4883.gd61b0886.el9.x86_6 13/35 2024-09-12T07:16:34.184 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4883.gd61b0886.el9.x86_6 13/35 2024-09-12T07:16:34.218 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-12T07:16:34.329 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-12T07:16:34.361 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-12T07:16:34.402 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-12T07:16:34.443 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-12T07:16:34.481 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-12T07:16:34.522 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-12T07:16:34.574 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-12T07:16:34.920 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-12T07:16:34.968 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:19.3.0-4883.gd61b0886.el9.x86_64 23/35 2024-09-12T07:16:35.061 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:19.3.0-4883.gd61b0886.el9.x86_64 23/35 2024-09-12T07:16:35.089 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:19.3.0-4883.gd61b0886.el9.x86_64 24/35 2024-09-12T07:16:35.126 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-12T07:16:35.587 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.gd61b0886.el9.x86_64 30/35 2024-09-12T07:16:35.621 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:19.3.0-4883.gd61b0886.el9.x86_64 30/35 2024-09-12T07:16:35.644 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-12T07:16:35.671 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:19.3.0-4883.gd61b0886.el9.x86_64 27/35 2024-09-12T07:16:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:19.3.0-4883.gd61b0886.el9.x86_64 27/35 2024-09-12T07:16:35.823 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:19.3.0-4883.gd61b0886.el9.x86_64 28/35 2024-09-12T07:16:35.856 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:19.3.0-4883.gd61b0886.el9.x8 29/35 2024-09-12T07:16:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.gd61b0886.el9.x86_64 30/35 2024-09-12T07:16:36.267 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:19.3.0-4883.gd61b0886.el9.x86_64 31/35 2024-09-12T07:16:36.359 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:19.3.0-4883.gd61b0886.el9.x86_64 31/35 2024-09-12T07:16:36.360 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T07:16:36.360 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T07:16:36.360 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:16:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:19.3.0-4883.gd61b0886.el9.x86_64 32/35 2024-09-12T07:16:38.481 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.gd61b0886.el9.x86_64 30/35 2024-09-12T07:16:38.528 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:19.3.0-4883.gd61b0886.el9.x86_64 30/35 2024-09-12T07:16:39.118 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.gd61b0886.el9.x86_64 30/35 2024-09-12T07:16:39.174 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:19.3.0-4883.gd61b0886.el9.x86_64 31/35 2024-09-12T07:16:39.265 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:19.3.0-4883.gd61b0886.el9.x86_64 31/35 2024-09-12T07:16:39.265 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T07:16:39.265 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T07:16:39.265 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:16:39.292 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:19.3.0-4883.gd61b0886.el9.x86_64 32/35 2024-09-12T07:16:46.705 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4883.gd61b0886.el9.x86_64 32/35 2024-09-12T07:16:46.706 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-09-12T07:16:46.706 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-09-12T07:16:46.706 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-09-12T07:16:46.706 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-09-12T07:16:46.706 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-09-12T07:16:46.706 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /root 2024-09-12T07:16:46.706 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-09-12T07:16:46.706 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:16:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:19.3.0-4883.gd61b0886.el9.x86_64 33/35 2024-09-12T07:16:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4883.gd61b0886.el9.x86_64 33/35 2024-09-12T07:16:47.781 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:16:47.781 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T07:16:47.781 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T07:16:47.782 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T07:16:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:16:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-12T07:16:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-12T07:16:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-12T07:16:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-12T07:16:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:19.3.0-4883.gd61b0886.el9.x86_64 1/35 2024-09-12T07:16:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:19.3.0-4883.gd61b0886.el9.x86_64 2/35 2024-09-12T07:16:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:19.3.0-4883.gd61b0886.el9.x86_64 3/35 2024-09-12T07:16:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:19.3.0-4883.gd61b0886.el9.x86_64 4/35 2024-09-12T07:16:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:19.3.0-4883.gd61b0886.el9.x86_64 5/35 2024-09-12T07:16:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:19.3.0-4883.gd61b0886.el9.x86_6 6/35 2024-09-12T07:16:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:19.3.0-4883.gd61b0886.el9.x86_64 7/35 2024-09-12T07:16:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4883.gd61b0886.el9. 8/35 2024-09-12T07:16:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:19.3.0-4883.gd61b0886.el9.x8 9/35 2024-09-12T07:16:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:19.3.0-4883.gd61b0886.el9.x86_64 10/35 2024-09-12T07:16:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:19.3.0-4883.gd61b0886.el9.x86_64 11/35 2024-09-12T07:16:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:19.3.0-4883.gd61b0886.el9.x86_64 12/35 2024-09-12T07:16:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:19.3.0-4883.gd61b0886.el9.x86_64 13/35 2024-09-12T07:16:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-12T07:16:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-12T07:16:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-12T07:16:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-12T07:16:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-12T07:16:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-12T07:16:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-12T07:16:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-12T07:16:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-12T07:16:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-12T07:16:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-12T07:16:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-12T07:16:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-12T07:16:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-12T07:16:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-12T07:16:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-12T07:16:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-12T07:16:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-12T07:16:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:19.3.0-4883.gd61b0886.el9.x86_64 32/35 2024-09-12T07:16:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-12T07:16:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:19.3.0-4883.gd61b0886.el9.x86_64 34/35 2024-09-12T07:16:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4883.gd61b0886.el9.x86_64 32/35 2024-09-12T07:16:49.654 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-09-12T07:16:49.654 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-09-12T07:16:49.654 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-09-12T07:16:49.654 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-09-12T07:16:49.655 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-09-12T07:16:49.655 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2024-09-12T07:16:49.655 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-09-12T07:16:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:16:50.078 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T07:16:50.079 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.080 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.081 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.081 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T07:16:50.081 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.081 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.081 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:50.081 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T07:16:50.081 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T07:16:50.081 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T07:16:50.081 INFO:teuthology.orchestra.run.smithi046.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T07:16:50.081 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T07:16:50.081 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:16:50.081 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:16:50.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-test 2024-09-12T07:16:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:19.3.0-4883.gd61b0886.el9.x86_64 33/35 2024-09-12T07:16:50.727 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4883.gd61b0886.el9.x86_64 33/35 2024-09-12T07:16:50.727 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:16:50.727 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T07:16:50.728 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T07:16:50.728 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T07:16:50.728 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:16:50.729 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-12T07:16:50.820 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-12T07:16:50.820 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-12T07:16:51.082 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:30 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:16:51.220 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:16:51.221 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:16:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-12T07:16:51.222 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:16:51.222 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:16:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 50 M 2024-09-12T07:16:51.222 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-12T07:16:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 161 k 2024-09-12T07:16:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T07:16:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T07:16:51.222 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:16:51.223 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:16:51.223 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:16:51.223 INFO:teuthology.orchestra.run.smithi046.stdout:Install 4 Packages 2024-09-12T07:16:51.223 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:16:51.223 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 51 M 2024-09-12T07:16:51.223 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 217 M 2024-09-12T07:16:51.223 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:16:51.686 INFO:teuthology.orchestra.run.smithi046.stdout:(1/4): libcephsqlite-19.3.0-4883.gd61b0886.el9. 787 kB/s | 161 kB 00:00 2024-09-12T07:16:51.737 INFO:teuthology.orchestra.run.smithi046.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-12T07:16:51.845 INFO:teuthology.orchestra.run.smithi046.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 401 kB/s | 64 kB 00:00 2024-09-12T07:16:52.055 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-12T07:16:52.055 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:19.3.0-4883.gd61b0886.el9.x86_64 1/35 2024-09-12T07:16:52.055 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:19.3.0-4883.gd61b0886.el9.x86_64 2/35 2024-09-12T07:16:52.055 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:19.3.0-4883.gd61b0886.el9.x86_64 3/35 2024-09-12T07:16:52.055 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:19.3.0-4883.gd61b0886.el9.x86_64 4/35 2024-09-12T07:16:52.055 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:19.3.0-4883.gd61b0886.el9.x86_64 5/35 2024-09-12T07:16:52.055 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:19.3.0-4883.gd61b0886.el9.x86_6 6/35 2024-09-12T07:16:52.055 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:19.3.0-4883.gd61b0886.el9.x86_64 7/35 2024-09-12T07:16:52.055 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4883.gd61b0886.el9. 8/35 2024-09-12T07:16:52.056 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:19.3.0-4883.gd61b0886.el9.x8 9/35 2024-09-12T07:16:52.056 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:19.3.0-4883.gd61b0886.el9.x86_64 10/35 2024-09-12T07:16:52.056 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:19.3.0-4883.gd61b0886.el9.x86_64 11/35 2024-09-12T07:16:52.056 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:19.3.0-4883.gd61b0886.el9.x86_64 12/35 2024-09-12T07:16:52.056 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:19.3.0-4883.gd61b0886.el9.x86_64 13/35 2024-09-12T07:16:52.056 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-12T07:16:52.056 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-12T07:16:52.056 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-12T07:16:52.056 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-12T07:16:52.056 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-12T07:16:52.056 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-12T07:16:52.056 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-12T07:16:52.057 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-12T07:16:52.057 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-12T07:16:52.057 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-12T07:16:52.057 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-12T07:16:52.057 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-12T07:16:52.057 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-12T07:16:52.057 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-12T07:16:52.057 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-12T07:16:52.057 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-12T07:16:52.058 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-12T07:16:52.058 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-12T07:16:52.058 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:19.3.0-4883.gd61b0886.el9.x86_64 32/35 2024-09-12T07:16:52.058 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-12T07:16:52.058 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:19.3.0-4883.gd61b0886.el9.x86_64 34/35 2024-09-12T07:16:52.864 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-12T07:16:52.864 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:16:52.864 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-09-12T07:16:52.864 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.864 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.864 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:16:52.864 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T07:16:52.864 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.864 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.865 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.865 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.865 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T07:16:52.865 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T07:16:52.865 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T07:16:52.865 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.865 INFO:teuthology.orchestra.run.smithi064.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-12T07:16:52.865 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T07:16:52.865 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T07:16:52.865 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T07:16:52.865 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.865 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T07:16:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T07:16:52.867 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T07:16:52.867 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T07:16:52.867 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T07:16:52.867 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:16:52.867 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:16:53.037 INFO:teuthology.orchestra.run.smithi046.stdout:(4/4): ceph-test-19.3.0-4883.gd61b0886.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-12T07:16:53.039 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:16:53.040 INFO:teuthology.orchestra.run.smithi046.stdout:Total 28 MB/s | 51 MB 00:01 2024-09-12T07:16:53.068 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:16:53.091 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:16:53.091 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:16:53.188 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-test 2024-09-12T07:16:53.524 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:16:53.525 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:16:53.775 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:28 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:16:53.923 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:16:53.923 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:16:53.923 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-12T07:16:53.923 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:16:53.923 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:16:53.923 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 50 M 2024-09-12T07:16:53.924 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-12T07:16:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 161 k 2024-09-12T07:16:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T07:16:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T07:16:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:16:53.924 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:16:53.924 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:16:53.924 INFO:teuthology.orchestra.run.smithi064.stdout:Install 4 Packages 2024-09-12T07:16:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:16:53.925 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 51 M 2024-09-12T07:16:53.925 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 217 M 2024-09-12T07:16:53.925 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:16:54.092 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:16:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-12T07:16:54.209 INFO:teuthology.orchestra.run.smithi064.stdout:(1/4): libcephsqlite-19.3.0-4883.gd61b0886.el9. 705 kB/s | 161 kB 00:00 2024-09-12T07:16:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-12T07:16:54.264 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:19.3.0-4883.gd61b0886.el9.x86_64 3/4 2024-09-12T07:16:54.851 INFO:teuthology.orchestra.run.smithi064.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 349 kB/s | 304 kB 00:00 2024-09-12T07:16:54.884 INFO:teuthology.orchestra.run.smithi064.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 94 kB/s | 64 kB 00:00 2024-09-12T07:16:55.534 INFO:teuthology.orchestra.run.smithi064.stdout:(4/4): ceph-test-19.3.0-4883.gd61b0886.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-12T07:16:55.537 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:16:55.537 INFO:teuthology.orchestra.run.smithi064.stdout:Total 31 MB/s | 51 MB 00:01 2024-09-12T07:16:55.565 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:16:55.589 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:16:55.589 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:16:56.026 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:16:56.027 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:16:56.588 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:16:56.662 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-12T07:16:56.700 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-12T07:16:56.728 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:19.3.0-4883.gd61b0886.el9.x86_64 3/4 2024-09-12T07:16:59.304 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4883.gd61b0886.el9.x86_64 3/4 2024-09-12T07:16:59.391 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:19.3.0-4883.gd61b0886.el9.x86_64 4/4 2024-09-12T07:17:01.789 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4883.gd61b0886.el9.x86_64 3/4 2024-09-12T07:17:01.849 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-test-2:19.3.0-4883.gd61b0886.el9.x86_64 4/4 2024-09-12T07:17:01.849 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:19.3.0-4883.gd61b0886.el9.x86_64 1/4 2024-09-12T07:17:01.849 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:19.3.0-4883.gd61b0886.el9.x86_64 2/4 2024-09-12T07:17:01.849 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-12T07:17:01.879 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:19.3.0-4883.gd61b0886.el9.x86_64 4/4 2024-09-12T07:17:02.186 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-12T07:17:02.186 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:02.186 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:17:02.186 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:02.186 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:02.186 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T07:17:02.186 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T07:17:02.187 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:02.187 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:17:02.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph 2024-09-12T07:17:03.075 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:42 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:17:03.215 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:17:03.218 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-12T07:17:03.218 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:03.218 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:17:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 6.6 k 2024-09-12T07:17:03.218 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-12T07:17:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 2.4 M 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 1.5 M 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 291 k 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 4.8 M 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 17 M 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T07:17:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T07:17:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:03.221 INFO:teuthology.orchestra.run.smithi046.stdout:Install 37 Packages 2024-09-12T07:17:03.222 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:03.222 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 31 M 2024-09-12T07:17:03.222 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 109 M 2024-09-12T07:17:03.222 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:17:03.689 INFO:teuthology.orchestra.run.smithi046.stdout:(1/37): ceph-19.3.0-4883.gd61b0886.el9.x86_64.r 45 kB/s | 6.6 kB 00:00 2024-09-12T07:17:03.864 INFO:teuthology.orchestra.run.smithi046.stdout:(2/37): ceph-mgr-19.3.0-4883.gd61b0886.el9.x86_ 4.7 MB/s | 1.5 MB 00:00 2024-09-12T07:17:03.918 INFO:teuthology.orchestra.run.smithi046.stdout:(3/37): ceph-mds-19.3.0-4883.gd61b0886.el9.x86_ 6.3 MB/s | 2.4 MB 00:00 2024-09-12T07:17:04.027 INFO:teuthology.orchestra.run.smithi046.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4883.gd61b 2.6 MB/s | 291 kB 00:00 2024-09-12T07:17:04.160 INFO:teuthology.orchestra.run.smithi046.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-09-12T07:17:04.277 INFO:teuthology.orchestra.run.smithi046.stdout:(6/37): ceph-mon-19.3.0-4883.gd61b0886.el9.x86_ 8.1 MB/s | 4.8 MB 00:00 2024-09-12T07:17:04.385 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-test-2:19.3.0-4883.gd61b0886.el9.x86_64 4/4 2024-09-12T07:17:04.385 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:19.3.0-4883.gd61b0886.el9.x86_64 1/4 2024-09-12T07:17:04.385 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:19.3.0-4883.gd61b0886.el9.x86_64 2/4 2024-09-12T07:17:04.385 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-12T07:17:04.461 INFO:teuthology.orchestra.run.smithi046.stdout:(7/37): ceph-osd-19.3.0-4883.gd61b0886.el9.x86_ 29 MB/s | 17 MB 00:00 2024-09-12T07:17:04.536 INFO:teuthology.orchestra.run.smithi046.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 3.3 MB/s | 1.2 MB 00:00 2024-09-12T07:17:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 375 kB/s | 106 kB 00:00 2024-09-12T07:17:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:(10/37): python3-pycparser-2.20-6.el9.noarch.rp 1.1 MB/s | 135 kB 00:00 2024-09-12T07:17:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:(11/37): python3-requests-2.25.1-8.el9.noarch.r 1.6 MB/s | 125 kB 00:00 2024-09-12T07:17:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-09-12T07:17:04.746 INFO:teuthology.orchestra.run.smithi046.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 321 kB/s | 35 kB 00:00 2024-09-12T07:17:04.796 INFO:teuthology.orchestra.run.smithi046.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 937 kB/s | 172 kB 00:00 2024-09-12T07:17:04.821 INFO:teuthology.orchestra.run.smithi046.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 804 kB/s | 188 kB 00:00 2024-09-12T07:17:04.847 INFO:teuthology.orchestra.run.smithi046.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 768 kB/s | 77 kB 00:00 2024-09-12T07:17:04.855 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-12T07:17:04.855 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:04.855 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:17:04.855 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:04.856 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:04.856 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T07:17:04.856 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T07:17:04.856 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:04.856 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:17:04.872 INFO:teuthology.orchestra.run.smithi046.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-12T07:17:04.897 INFO:teuthology.orchestra.run.smithi046.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-09-12T07:17:04.947 INFO:teuthology.orchestra.run.smithi046.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-12T07:17:04.972 INFO:teuthology.orchestra.run.smithi046.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 578 kB/s | 43 kB 00:00 2024-09-12T07:17:04.998 INFO:teuthology.orchestra.run.smithi046.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-09-12T07:17:05.023 INFO:teuthology.orchestra.run.smithi046.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-12T07:17:05.048 INFO:teuthology.orchestra.run.smithi046.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 214 kB/s | 11 kB 00:00 2024-09-12T07:17:05.074 INFO:teuthology.orchestra.run.smithi046.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 350 kB/s | 18 kB 00:00 2024-09-12T07:17:05.099 INFO:teuthology.orchestra.run.smithi046.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-12T07:17:05.133 INFO:teuthology.orchestra.run.smithi046.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 331 kB/s | 19 kB 00:00 2024-09-12T07:17:05.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph 2024-09-12T07:17:05.158 INFO:teuthology.orchestra.run.smithi046.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 334 kB/s | 20 kB 00:00 2024-09-12T07:17:05.183 INFO:teuthology.orchestra.run.smithi046.stdout:(28/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-09-12T07:17:05.217 INFO:teuthology.orchestra.run.smithi046.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 551 kB/s | 46 kB 00:00 2024-09-12T07:17:05.243 INFO:teuthology.orchestra.run.smithi046.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 941 kB/s | 79 kB 00:00 2024-09-12T07:17:05.268 INFO:teuthology.orchestra.run.smithi046.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-12T07:17:05.293 INFO:teuthology.orchestra.run.smithi046.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-12T07:17:05.318 INFO:teuthology.orchestra.run.smithi046.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-12T07:17:05.344 INFO:teuthology.orchestra.run.smithi046.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-12T07:17:05.369 INFO:teuthology.orchestra.run.smithi046.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T07:17:05.394 INFO:teuthology.orchestra.run.smithi046.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-09-12T07:17:05.428 INFO:teuthology.orchestra.run.smithi046.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-12T07:17:05.428 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:05.428 INFO:teuthology.orchestra.run.smithi046.stdout:Total 14 MB/s | 31 MB 00:02 2024-09-12T07:17:05.586 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:17:05.630 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:17:05.630 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:17:05.731 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:40 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:17:05.871 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:17:05.873 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:05.873 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-12T07:17:05.873 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:05.873 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 6.6 k 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 2.4 M 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 1.5 M 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 291 k 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 4.8 M 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 17 M 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T07:17:05.874 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T07:17:05.875 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T07:17:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:05.877 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:17:05.877 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:05.877 INFO:teuthology.orchestra.run.smithi064.stdout:Install 37 Packages 2024-09-12T07:17:05.877 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:05.877 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 31 M 2024-09-12T07:17:05.877 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 109 M 2024-09-12T07:17:05.877 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:17:06.118 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:17:06.118 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:17:06.749 INFO:teuthology.orchestra.run.smithi064.stdout:(1/37): ceph-19.3.0-4883.gd61b0886.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-12T07:17:06.925 INFO:teuthology.orchestra.run.smithi064.stdout:(2/37): ceph-mgr-19.3.0-4883.gd61b0886.el9.x86_ 4.9 MB/s | 1.5 MB 00:00 2024-09-12T07:17:06.975 INFO:teuthology.orchestra.run.smithi064.stdout:(3/37): ceph-mds-19.3.0-4883.gd61b0886.el9.x86_ 6.5 MB/s | 2.4 MB 00:00 2024-09-12T07:17:07.050 INFO:teuthology.orchestra.run.smithi064.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4883.gd61b 3.8 MB/s | 291 kB 00:00 2024-09-12T07:17:07.134 INFO:teuthology.orchestra.run.smithi064.stdout:(5/37): ceph-mon-19.3.0-4883.gd61b0886.el9.x86_ 12 MB/s | 4.8 MB 00:00 2024-09-12T07:17:07.259 INFO:teuthology.orchestra.run.smithi064.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-12T07:17:07.301 INFO:teuthology.orchestra.run.smithi064.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-12T07:17:07.351 INFO:teuthology.orchestra.run.smithi064.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 5.6 MB/s | 1.2 MB 00:00 2024-09-12T07:17:07.377 INFO:teuthology.orchestra.run.smithi064.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-09-12T07:17:07.402 INFO:teuthology.orchestra.run.smithi064.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-12T07:17:07.427 INFO:teuthology.orchestra.run.smithi064.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-12T07:17:07.515 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:17:07.581 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-12T07:17:07.594 INFO:teuthology.orchestra.run.smithi064.stdout:(12/37): ceph-osd-19.3.0-4883.gd61b0886.el9.x86 26 MB/s | 17 MB 00:00 2024-09-12T07:17:07.643 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-12T07:17:07.683 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-12T07:17:07.686 INFO:teuthology.orchestra.run.smithi064.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 380 kB/s | 35 kB 00:00 2024-09-12T07:17:07.711 INFO:teuthology.orchestra.run.smithi064.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 610 kB/s | 188 kB 00:00 2024-09-12T07:17:07.754 INFO:teuthology.orchestra.run.smithi064.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 528 kB/s | 172 kB 00:00 2024-09-12T07:17:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-12T07:17:07.779 INFO:teuthology.orchestra.run.smithi064.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 839 kB/s | 77 kB 00:00 2024-09-12T07:17:07.804 INFO:teuthology.orchestra.run.smithi064.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 552 kB/s | 51 kB 00:00 2024-09-12T07:17:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-12T07:17:07.829 INFO:teuthology.orchestra.run.smithi064.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-09-12T07:17:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-12T07:17:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-12T07:17:08.031 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-12T07:17:08.077 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-12T07:17:08.096 INFO:teuthology.orchestra.run.smithi064.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 70 kB/s | 22 kB 00:00 2024-09-12T07:17:08.112 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-12T07:17:08.150 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-12T07:17:08.185 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-12T07:17:08.221 INFO:teuthology.orchestra.run.smithi064.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 111 kB/s | 43 kB 00:00 2024-09-12T07:17:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-12T07:17:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-12T07:17:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-12T07:17:08.347 INFO:teuthology.orchestra.run.smithi064.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 279 kB/s | 151 kB 00:00 2024-09-12T07:17:08.371 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-12T07:17:08.421 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-12T07:17:08.447 INFO:teuthology.orchestra.run.smithi064.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-09-12T07:17:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-12T07:17:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-12T07:17:08.556 INFO:teuthology.orchestra.run.smithi064.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 163 kB/s | 18 kB 00:00 2024-09-12T07:17:08.581 INFO:teuthology.orchestra.run.smithi064.stdout:(24/37): python3-cherrypy-18.6.1-2.el9.noarch.r 997 kB/s | 358 kB 00:00 2024-09-12T07:17:08.631 INFO:teuthology.orchestra.run.smithi064.stdout:(25/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 321 kB/s | 172 kB 00:00 2024-09-12T07:17:08.656 INFO:teuthology.orchestra.run.smithi064.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 230 kB/s | 23 kB 00:00 2024-09-12T07:17:08.682 INFO:teuthology.orchestra.run.smithi064.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 193 kB/s | 19 kB 00:00 2024-09-12T07:17:08.698 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-12T07:17:08.732 INFO:teuthology.orchestra.run.smithi064.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 196 kB/s | 20 kB 00:00 2024-09-12T07:17:08.766 INFO:teuthology.orchestra.run.smithi064.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 425 kB/s | 46 kB 00:00 2024-09-12T07:17:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-12T07:17:08.791 INFO:teuthology.orchestra.run.smithi064.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 724 kB/s | 79 kB 00:00 2024-09-12T07:17:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-12T07:17:08.866 INFO:teuthology.orchestra.run.smithi064.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 164 kB/s | 16 kB 00:00 2024-09-12T07:17:08.900 INFO:teuthology.orchestra.run.smithi064.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 829 kB/s | 90 kB 00:00 2024-09-12T07:17:08.901 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-12T07:17:08.933 INFO:teuthology.orchestra.run.smithi064.stdout:(33/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-12T07:17:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-12T07:17:09.000 INFO:teuthology.orchestra.run.smithi064.stdout:(34/37): python3-tempora-5.0.0-2.el9.noarch.rpm 267 kB/s | 36 kB 00:00 2024-09-12T07:17:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-12T07:17:09.034 INFO:teuthology.orchestra.run.smithi064.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-12T07:17:09.101 INFO:teuthology.orchestra.run.smithi064.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 199 kB/s | 20 kB 00:00 2024-09-12T07:17:09.126 INFO:teuthology.orchestra.run.smithi064.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.2 MB/s | 427 kB 00:00 2024-09-12T07:17:09.131 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:09.131 INFO:teuthology.orchestra.run.smithi064.stdout:Total 9.4 MB/s | 31 MB 00:03 2024-09-12T07:17:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-12T07:17:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-12T07:17:09.287 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:17:09.332 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:17:09.332 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:17:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-12T07:17:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-12T07:17:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-12T07:17:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-12T07:17:09.810 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4883.gd61b0886.el9. 32/37 2024-09-12T07:17:09.840 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:19.3.0-4883.gd61b0886.el9.x86_64 33/37 2024-09-12T07:17:09.867 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4883.gd61b0886.el9.x86_64 33/37 2024-09-12T07:17:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:17:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T07:17:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T07:17:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T07:17:09.867 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:09.877 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:17:09.877 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:17:11.398 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:17:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:19.3.0-4883.gd61b0886.el9.x86_64 34/37 2024-09-12T07:17:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:19.3.0-4883.gd61b0886.el9.x86_64 34/37 2024-09-12T07:17:11.454 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:17:11.454 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T07:17:11.454 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T07:17:11.454 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T07:17:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:11.462 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-12T07:17:11.506 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-12T07:17:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-12T07:17:11.624 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-12T07:17:11.673 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-12T07:17:11.715 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-12T07:17:11.819 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-12T07:17:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-12T07:17:11.925 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:19.3.0-4883.gd61b0886.el9.x86_64 35/37 2024-09-12T07:17:11.950 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:19.3.0-4883.gd61b0886.el9.x86_64 35/37 2024-09-12T07:17:11.950 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:17:11.950 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T07:17:11.950 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T07:17:11.951 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T07:17:11.951 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:11.952 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-12T07:17:11.986 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-12T07:17:12.016 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-12T07:17:12.051 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-12T07:17:12.092 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-12T07:17:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-12T07:17:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-12T07:17:12.192 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:19.3.0-4883.gd61b0886.el9.x86_64 36/37 2024-09-12T07:17:12.217 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:19.3.0-4883.gd61b0886.el9.x86_64 36/37 2024-09-12T07:17:12.217 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:17:12.217 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T07:17:12.217 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T07:17:12.217 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T07:17:12.218 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-12T07:17:12.268 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-12T07:17:12.283 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:19.3.0-4883.gd61b0886.el9.x86_64 37/37 2024-09-12T07:17:12.308 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-12T07:17:12.384 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-12T07:17:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-12T07:17:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-12T07:17:12.920 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-12T07:17:12.958 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-12T07:17:13.016 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-12T07:17:13.080 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-12T07:17:13.275 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-12T07:17:13.328 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-12T07:17:13.409 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-12T07:17:13.527 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-12T07:17:13.686 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-12T07:17:13.821 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-12T07:17:13.991 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4883.gd61b0886.el9. 32/37 2024-09-12T07:17:14.020 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:19.3.0-4883.gd61b0886.el9.x86_64 33/37 2024-09-12T07:17:14.047 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4883.gd61b0886.el9.x86_64 33/37 2024-09-12T07:17:14.047 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:17:14.048 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T07:17:14.048 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T07:17:14.048 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T07:17:14.048 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-2:19.3.0-4883.gd61b0886.el9.x86_64 37/37 2024-09-12T07:17:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:19.3.0-4883.gd61b0886.el9.x86_64 1/37 2024-09-12T07:17:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:19.3.0-4883.gd61b0886.el9.x86_64 2/37 2024-09-12T07:17:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:19.3.0-4883.gd61b0886.el9.x86_64 3/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:19.3.0-4883.gd61b0886.el9.x86_64 4/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:19.3.0-4883.gd61b0886.el9.x86_64 5/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4883.gd61b0886.el9. 6/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-12T07:17:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-12T07:17:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-12T07:17:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-12T07:17:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-12T07:17:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-12T07:17:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-12T07:17:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-12T07:17:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-12T07:17:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-12T07:17:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-12T07:17:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-12T07:17:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-12T07:17:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-12T07:17:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-12T07:17:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-12T07:17:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-12T07:17:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-12T07:17:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-12T07:17:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-12T07:17:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-12T07:17:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-12T07:17:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-12T07:17:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:15.206 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:17:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T07:17:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T07:17:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T07:17:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T07:17:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T07:17:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T07:17:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T07:17:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T07:17:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T07:17:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T07:17:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:15.209 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:17:15.575 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-base 2024-09-12T07:17:15.630 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:19.3.0-4883.gd61b0886.el9.x86_64 34/37 2024-09-12T07:17:15.662 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:19.3.0-4883.gd61b0886.el9.x86_64 34/37 2024-09-12T07:17:15.662 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:17:15.662 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T07:17:15.662 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T07:17:15.662 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T07:17:15.662 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:16.148 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:19.3.0-4883.gd61b0886.el9.x86_64 35/37 2024-09-12T07:17:16.165 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:55 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:17:16.173 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:19.3.0-4883.gd61b0886.el9.x86_64 35/37 2024-09-12T07:17:16.173 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:17:16.174 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T07:17:16.174 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T07:17:16.174 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T07:17:16.174 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:16.259 INFO:teuthology.orchestra.run.smithi046.stdout:Package ceph-base-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:17:16.305 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:17:16.306 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-12T07:17:16.306 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:17:16.375 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install cephadm 2024-09-12T07:17:16.415 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:19.3.0-4883.gd61b0886.el9.x86_64 36/37 2024-09-12T07:17:16.440 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:19.3.0-4883.gd61b0886.el9.x86_64 36/37 2024-09-12T07:17:16.441 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:17:16.441 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T07:17:16.441 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T07:17:16.441 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T07:17:16.441 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:16.516 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:19.3.0-4883.gd61b0886.el9.x86_64 37/37 2024-09-12T07:17:16.970 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:55 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:17:17.112 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:17:17.112 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-12T07:17:17.113 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:17.113 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:17:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 784 k 2024-09-12T07:17:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:17.113 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:17:17.113 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:17.113 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-12T07:17:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:17.113 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 784 k 2024-09-12T07:17:17.113 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 791 k 2024-09-12T07:17:17.113 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:17:17.384 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm-19.3.0-4883.gd61b0886.el9.noarch.rpm 2.8 MB/s | 784 kB 00:00 2024-09-12T07:17:17.385 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:17.385 INFO:teuthology.orchestra.run.smithi046.stdout:Total 2.8 MB/s | 784 kB 00:00 2024-09-12T07:17:17.385 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:17:17.392 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:17:17.392 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:17:17.398 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:17:17.398 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:17:17.470 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:17:17.939 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:17:18.033 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:17:18.433 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:17:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-2:19.3.0-4883.gd61b0886.el9.x86_64 37/37 2024-09-12T07:17:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:19.3.0-4883.gd61b0886.el9.x86_64 1/37 2024-09-12T07:17:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:19.3.0-4883.gd61b0886.el9.x86_64 2/37 2024-09-12T07:17:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:19.3.0-4883.gd61b0886.el9.x86_64 3/37 2024-09-12T07:17:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:19.3.0-4883.gd61b0886.el9.x86_64 4/37 2024-09-12T07:17:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:19.3.0-4883.gd61b0886.el9.x86_64 5/37 2024-09-12T07:17:18.476 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4883.gd61b0886.el9. 6/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-12T07:17:18.477 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-12T07:17:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-12T07:17:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-12T07:17:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-12T07:17:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-12T07:17:18.478 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-12T07:17:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-12T07:17:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-12T07:17:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-12T07:17:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-12T07:17:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-12T07:17:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-12T07:17:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-12T07:17:18.479 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-12T07:17:18.480 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-12T07:17:18.480 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-12T07:17:18.480 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-12T07:17:18.703 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:17:18.703 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:18.703 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:17:18.703 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:18.703 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:18.703 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:17:18.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-immutable-object-cache 2024-09-12T07:17:19.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-12T07:17:19.413 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T07:17:19.414 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T07:17:19.415 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:19.416 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:17:19.516 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:58 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:17:19.655 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:17:19.656 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:19.656 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-09-12T07:17:19.656 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:19.656 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:17:19.656 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 145 k 2024-09-12T07:17:19.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:19.656 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:17:19.656 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:19.657 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-12T07:17:19.657 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:19.657 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 145 k 2024-09-12T07:17:19.657 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 432 k 2024-09-12T07:17:19.657 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:17:19.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-base 2024-09-12T07:17:19.866 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-immutable-object-cache-19.3.0-4883.gd61b08 693 kB/s | 145 kB 00:00 2024-09-12T07:17:19.867 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:19.867 INFO:teuthology.orchestra.run.smithi046.stdout:Total 688 kB/s | 145 kB 00:00 2024-09-12T07:17:19.867 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:17:19.875 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:17:19.875 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:17:19.936 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:17:19.937 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:17:20.086 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:17:20.159 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4883.gd61b0886. 1/1 2024-09-12T07:17:20.186 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4883.gd61b0886. 1/1 2024-09-12T07:17:20.186 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:17:20.186 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T07:17:20.186 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:20.383 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:55 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:17:20.478 INFO:teuthology.orchestra.run.smithi064.stdout:Package ceph-base-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:17:20.522 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:17:20.523 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-09-12T07:17:20.523 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:17:20.602 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install cephadm 2024-09-12T07:17:21.119 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4883.gd61b0886. 1/1 2024-09-12T07:17:21.120 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:21.120 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:17:21.120 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:21.120 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:21.120 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:17:21.195 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:56 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:17:21.332 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:17:21.333 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:21.333 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-12T07:17:21.334 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:21.334 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:17:21.334 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 784 k 2024-09-12T07:17:21.334 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:21.334 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:17:21.334 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:21.335 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-09-12T07:17:21.335 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:21.335 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 784 k 2024-09-12T07:17:21.335 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 791 k 2024-09-12T07:17:21.335 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:17:21.355 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr 2024-09-12T07:17:21.607 INFO:teuthology.orchestra.run.smithi064.stdout:cephadm-19.3.0-4883.gd61b0886.el9.noarch.rpm 2.8 MB/s | 784 kB 00:00 2024-09-12T07:17:21.607 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:21.608 INFO:teuthology.orchestra.run.smithi064.stdout:Total 2.8 MB/s | 784 kB 00:00 2024-09-12T07:17:21.608 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:17:21.614 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:17:21.614 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:17:21.620 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:17:21.620 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:17:21.697 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:17:21.946 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:00 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:17:22.039 INFO:teuthology.orchestra.run.smithi046.stdout:Package ceph-mgr-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:17:22.084 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:17:22.085 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-12T07:17:22.085 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:17:22.146 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:17:22.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-dashboard 2024-09-12T07:17:22.246 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:17:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:17:22.743 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:01 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:17:22.854 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:17:22.854 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:22.854 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:17:22.854 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:22.854 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:22.854 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:17:22.880 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:17:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-12T07:17:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:17:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 14 M 2024-09-12T07:17:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-12T07:17:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T07:17:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 34 k 2024-09-12T07:17:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 16 k 2024-09-12T07:17:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T07:17:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T07:17:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T07:17:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T07:17:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T07:17:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T07:17:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T07:17:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T07:17:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:22.883 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:17:22.883 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:22.883 INFO:teuthology.orchestra.run.smithi046.stdout:Install 12 Packages 2024-09-12T07:17:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:22.884 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 19 M 2024-09-12T07:17:22.884 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 159 M 2024-09-12T07:17:22.884 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:17:23.071 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-immutable-object-cache 2024-09-12T07:17:23.481 INFO:teuthology.orchestra.run.smithi046.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4883.gd6 214 kB/s | 34 kB 00:00 2024-09-12T07:17:23.506 INFO:teuthology.orchestra.run.smithi046.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4883.gd61 89 kB/s | 16 kB 00:00 2024-09-12T07:17:23.657 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:58 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:17:23.797 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:17:23.798 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:23.798 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-09-12T07:17:23.798 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:23.799 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:17:23.799 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 145 k 2024-09-12T07:17:23.799 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:23.799 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:17:23.799 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:23.799 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-09-12T07:17:23.799 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:23.799 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 145 k 2024-09-12T07:17:23.800 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 432 k 2024-09-12T07:17:23.800 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:17:23.982 INFO:teuthology.orchestra.run.smithi046.stdout:(3/12): ceph-mgr-dashboard-19.3.0-4883.gd61b088 21 MB/s | 14 MB 00:00 2024-09-12T07:17:24.013 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-immutable-object-cache-19.3.0-4883.gd61b08 678 kB/s | 145 kB 00:00 2024-09-12T07:17:24.014 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:24.014 INFO:teuthology.orchestra.run.smithi064.stdout:Total 673 kB/s | 145 kB 00:00 2024-09-12T07:17:24.014 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:17:24.022 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:17:24.022 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:17:24.082 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:17:24.083 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:17:24.165 INFO:teuthology.orchestra.run.smithi046.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 4.6 MB/s | 863 kB 00:00 2024-09-12T07:17:24.227 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:17:24.257 INFO:teuthology.orchestra.run.smithi046.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 5.8 MB/s | 548 kB 00:00 2024-09-12T07:17:24.289 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4883.gd61b0886. 1/1 2024-09-12T07:17:24.291 INFO:teuthology.orchestra.run.smithi046.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 587 kB/s | 19 kB 00:00 2024-09-12T07:17:24.316 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4883.gd61b0886. 1/1 2024-09-12T07:17:24.316 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:17:24.316 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T07:17:24.316 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:24.358 INFO:teuthology.orchestra.run.smithi046.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-12T07:17:24.408 INFO:teuthology.orchestra.run.smithi046.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-09-12T07:17:24.441 INFO:teuthology.orchestra.run.smithi046.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 928 kB/s | 31 kB 00:00 2024-09-12T07:17:24.475 INFO:teuthology.orchestra.run.smithi046.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-09-12T07:17:24.542 INFO:teuthology.orchestra.run.smithi046.stdout:(11/12): python3-protobuf-3.14.0-13.el9.noarch. 260 kB/s | 269 kB 00:01 2024-09-12T07:17:25.303 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4883.gd61b0886. 1/1 2024-09-12T07:17:25.303 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:25.303 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:17:25.303 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:17:25.303 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:25.303 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:17:25.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr 2024-09-12T07:17:25.651 INFO:teuthology.orchestra.run.smithi046.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 476 kB/s | 1.0 MB 00:02 2024-09-12T07:17:25.654 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:25.654 INFO:teuthology.orchestra.run.smithi046.stdout:Total 7.0 MB/s | 19 MB 00:02 2024-09-12T07:17:25.708 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:17:25.730 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:17:25.730 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:17:26.022 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:17:26.022 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:17:26.147 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:01 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:17:26.240 INFO:teuthology.orchestra.run.smithi064.stdout:Package ceph-mgr-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:17:26.286 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:17:26.287 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-09-12T07:17:26.287 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:17:26.372 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-dashboard 2024-09-12T07:17:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:17:26.735 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-12T07:17:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-12T07:17:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-12T07:17:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-12T07:17:26.948 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-12T07:17:26.955 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:01 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:17:27.088 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-12T07:17:27.094 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:17:27.095 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:27.095 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-12T07:17:27.095 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:27.095 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:17:27.095 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 14 M 2024-09-12T07:17:27.095 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-12T07:17:27.095 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T07:17:27.095 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 34 k 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 16 k 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:27.096 INFO:teuthology.orchestra.run.smithi064.stdout:Install 12 Packages 2024-09-12T07:17:27.097 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:27.097 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 19 M 2024-09-12T07:17:27.097 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 159 M 2024-09-12T07:17:27.097 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:17:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-12T07:17:27.245 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-12T07:17:27.272 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-12T07:17:27.305 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4883.gd61b0886.el9 10/12 2024-09-12T07:17:27.453 INFO:teuthology.orchestra.run.smithi064.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4883.gd6 226 kB/s | 34 kB 00:00 2024-09-12T07:17:27.478 INFO:teuthology.orchestra.run.smithi064.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4883.gd61 93 kB/s | 16 kB 00:00 2024-09-12T07:17:27.979 INFO:teuthology.orchestra.run.smithi064.stdout:(3/12): ceph-mgr-dashboard-19.3.0-4883.gd61b088 21 MB/s | 14 MB 00:00 2024-09-12T07:17:28.545 INFO:teuthology.orchestra.run.smithi064.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 252 kB/s | 269 kB 00:01 2024-09-12T07:17:28.721 INFO:teuthology.orchestra.run.smithi064.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.1 MB/s | 548 kB 00:00 2024-09-12T07:17:28.754 INFO:teuthology.orchestra.run.smithi064.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 1.1 MB/s | 863 kB 00:00 2024-09-12T07:17:28.788 INFO:teuthology.orchestra.run.smithi064.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-12T07:17:28.872 INFO:teuthology.orchestra.run.smithi064.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 17 MB/s | 2.0 MB 00:00 2024-09-12T07:17:28.897 INFO:teuthology.orchestra.run.smithi064.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.3 MB/s | 144 kB 00:00 2024-09-12T07:17:28.947 INFO:teuthology.orchestra.run.smithi064.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 411 kB/s | 31 kB 00:00 2024-09-12T07:17:28.972 INFO:teuthology.orchestra.run.smithi064.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-09-12T07:17:29.739 INFO:teuthology.orchestra.run.smithi064.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 452 kB/s | 1.0 MB 00:02 2024-09-12T07:17:29.744 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:29.744 INFO:teuthology.orchestra.run.smithi064.stdout:Total 7.3 MB/s | 19 MB 00:02 2024-09-12T07:17:29.797 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:17:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:17:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:17:30.106 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:17:30.106 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:17:30.705 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4883.gd61b0886.el 11/12 2024-09-12T07:17:30.724 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:17:30.768 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4883.gd61b0886.el9.noa 12/12 2024-09-12T07:17:30.807 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-12T07:17:30.850 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-12T07:17:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-12T07:17:30.936 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-12T07:17:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-12T07:17:31.176 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-12T07:17:31.267 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-12T07:17:31.324 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-12T07:17:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-12T07:17:31.385 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4883.gd61b0886.el9 10/12 2024-09-12T07:17:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4883.gd61b0886.el9.noa 12/12 2024-09-12T07:17:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4883.gd61b0886.el 1/12 2024-09-12T07:17:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4883.gd61b0886.el9.noa 2/12 2024-09-12T07:17:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4883.gd61b0886.el9 3/12 2024-09-12T07:17:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-12T07:17:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-12T07:17:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-12T07:17:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-12T07:17:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-12T07:17:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-12T07:17:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-12T07:17:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-12T07:17:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-12T07:17:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:32.918 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:17:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T07:17:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:32.918 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T07:17:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T07:17:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T07:17:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T07:17:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T07:17:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T07:17:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T07:17:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T07:17:32.919 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:32.919 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:17:33.179 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-12T07:17:33.765 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:12 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:17:33.905 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:17:33.906 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-09-12T07:17:33.906 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-12T07:17:33.906 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-09-12T07:17:33.907 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:17:33.907 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 7.4 M 2024-09-12T07:17:33.907 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-12T07:17:33.907 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T07:17:33.907 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T07:17:33.907 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T07:17:33.907 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T07:17:33.907 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T07:17:33.907 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T07:17:33.907 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T07:17:33.907 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T07:17:33.907 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T07:17:33.908 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T07:17:33.908 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T07:17:33.908 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:33.908 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:17:33.908 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-09-12T07:17:33.908 INFO:teuthology.orchestra.run.smithi046.stdout:Install 12 Packages 2024-09-12T07:17:33.908 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:33.909 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 38 M 2024-09-12T07:17:33.909 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 212 M 2024-09-12T07:17:33.909 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:17:34.741 INFO:teuthology.orchestra.run.smithi046.stdout:(1/12): ceph-mgr-diskprediction-local-19.3.0-48 16 MB/s | 7.4 MB 00:00 2024-09-12T07:17:34.743 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4883.gd61b0886.el 11/12 2024-09-12T07:17:34.805 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4883.gd61b0886.el9.noa 12/12 2024-09-12T07:17:34.900 INFO:teuthology.orchestra.run.smithi046.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 312 kB/s | 191 kB 00:00 2024-09-12T07:17:34.925 INFO:teuthology.orchestra.run.smithi046.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 178 kB/s | 33 kB 00:00 2024-09-12T07:17:35.000 INFO:teuthology.orchestra.run.smithi046.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 242 kB/s | 18 kB 00:00 2024-09-12T07:17:35.126 INFO:teuthology.orchestra.run.smithi046.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 308 kB/s | 38 kB 00:00 2024-09-12T07:17:35.359 INFO:teuthology.orchestra.run.smithi046.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 747 kB/s | 801 kB 00:01 2024-09-12T07:17:35.710 INFO:teuthology.orchestra.run.smithi046.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 700 kB/s | 245 kB 00:00 2024-09-12T07:17:36.143 INFO:teuthology.orchestra.run.smithi046.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.4 MB/s | 3.0 MB 00:01 2024-09-12T07:17:36.352 INFO:teuthology.orchestra.run.smithi046.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.2 MB/s | 250 kB 00:00 2024-09-12T07:17:36.577 INFO:teuthology.orchestra.run.smithi046.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.4 MB/s | 4.9 MB 00:01 2024-09-12T07:17:36.719 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4883.gd61b0886.el9.noa 12/12 2024-09-12T07:17:36.720 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4883.gd61b0886.el 1/12 2024-09-12T07:17:36.720 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4883.gd61b0886.el9.noa 2/12 2024-09-12T07:17:36.720 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4883.gd61b0886.el9 3/12 2024-09-12T07:17:36.720 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-12T07:17:36.720 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-12T07:17:36.720 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-12T07:17:36.720 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-12T07:17:36.720 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-12T07:17:36.720 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-12T07:17:36.720 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-12T07:17:36.720 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-12T07:17:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-12T07:17:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:17:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T07:17:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T07:17:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T07:17:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T07:17:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T07:17:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T07:17:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T07:17:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T07:17:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T07:17:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:17:37.312 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-12T07:17:37.901 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:12 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:17:38.003 INFO:teuthology.orchestra.run.smithi046.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.2 MB/s | 5.1 MB 00:02 2024-09-12T07:17:38.040 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:17:38.041 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-09-12T07:17:38.041 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-12T07:17:38.041 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-09-12T07:17:38.041 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:17:38.041 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 7.4 M 2024-09-12T07:17:38.041 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:17:38.042 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-09-12T07:17:38.043 INFO:teuthology.orchestra.run.smithi064.stdout:Install 12 Packages 2024-09-12T07:17:38.043 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:38.043 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 38 M 2024-09-12T07:17:38.043 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 212 M 2024-09-12T07:17:38.043 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:17:38.504 INFO:teuthology.orchestra.run.smithi064.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-12T07:17:38.546 INFO:teuthology.orchestra.run.smithi064.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-12T07:17:38.830 INFO:teuthology.orchestra.run.smithi064.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-48 16 MB/s | 7.4 MB 00:00 2024-09-12T07:17:39.113 INFO:teuthology.orchestra.run.smithi064.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 54 kB/s | 33 kB 00:00 2024-09-12T07:17:39.372 INFO:teuthology.orchestra.run.smithi064.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 34 kB/s | 18 kB 00:00 2024-09-12T07:17:39.789 INFO:teuthology.orchestra.run.smithi064.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.4 MB/s | 3.0 MB 00:01 2024-09-12T07:17:39.806 INFO:teuthology.orchestra.run.smithi064.stdout:(7/12): openblas-0.3.26-2.el9.x86_64.rpm 56 kB/s | 38 kB 00:00 2024-09-12T07:17:40.614 INFO:teuthology.orchestra.run.smithi064.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.0 MB/s | 4.9 MB 00:01 2024-09-12T07:17:40.640 INFO:teuthology.orchestra.run.smithi064.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 288 kB/s | 245 kB 00:00 2024-09-12T07:17:41.090 INFO:teuthology.orchestra.run.smithi064.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.0 MB/s | 5.1 MB 00:01 2024-09-12T07:17:41.568 INFO:teuthology.orchestra.run.smithi064.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 264 kB/s | 250 kB 00:00 2024-09-12T07:17:42.240 INFO:teuthology.orchestra.run.smithi064.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-12T07:17:42.241 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:42.242 INFO:teuthology.orchestra.run.smithi064.stdout:Total 9.1 MB/s | 38 MB 00:04 2024-09-12T07:17:42.612 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:17:42.641 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:17:42.641 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:17:43.089 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:17:43.089 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:17:43.925 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:17:44.003 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-12T07:17:44.064 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-12T07:17:44.170 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-12T07:17:44.207 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-12T07:17:44.398 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-12T07:17:44.431 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-12T07:17:44.526 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-12T07:17:44.727 INFO:teuthology.orchestra.run.smithi046.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.9 MB/s | 16 MB 00:08 2024-09-12T07:17:44.728 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:44.728 INFO:teuthology.orchestra.run.smithi046.stdout:Total 3.5 MB/s | 38 MB 00:10 2024-09-12T07:17:45.097 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:17:45.125 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:17:45.125 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:17:45.226 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-12T07:17:45.350 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-12T07:17:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:17:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:17:46.488 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:17:46.568 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-12T07:17:46.625 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-12T07:17:46.684 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-12T07:17:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-12T07:17:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-12T07:17:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-12T07:17:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-12T07:17:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-12T07:17:47.766 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-12T07:17:47.796 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4883.gd61b0 12/12 2024-09-12T07:17:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-12T07:17:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-12T07:17:49.318 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-12T07:17:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-12T07:17:50.392 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4883.gd61b0 12/12 2024-09-12T07:17:50.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4883.gd61b0 1/12 2024-09-12T07:17:50.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-12T07:17:50.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-12T07:17:50.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-12T07:17:50.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-12T07:17:50.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-12T07:17:50.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-12T07:17:50.393 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-12T07:17:50.393 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-12T07:17:50.393 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-12T07:17:50.393 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-12T07:17:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4883.gd61b0 12/12 2024-09-12T07:17:50.779 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-12T07:17:50.779 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:50.779 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:17:50.779 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:50.780 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:17:51.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-rook 2024-09-12T07:17:51.744 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:26 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:17:51.885 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:17:51.886 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:51.887 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-12T07:17:51.887 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:51.887 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:17:51.887 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 49 k 2024-09-12T07:17:51.887 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-12T07:17:51.887 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T07:17:51.887 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T07:17:51.887 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T07:17:51.887 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T07:17:51.887 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T07:17:51.887 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T07:17:51.887 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T07:17:51.888 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T07:17:51.888 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T07:17:51.888 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T07:17:51.888 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T07:17:51.888 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T07:17:51.888 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:51.888 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:17:51.888 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:51.888 INFO:teuthology.orchestra.run.smithi064.stdout:Install 13 Packages 2024-09-12T07:17:51.888 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:51.889 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 2.2 M 2024-09-12T07:17:51.889 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 26 M 2024-09-12T07:17:51.889 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:17:52.264 INFO:teuthology.orchestra.run.smithi064.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 203 kB/s | 26 kB 00:00 2024-09-12T07:17:52.289 INFO:teuthology.orchestra.run.smithi064.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 126 kB/s | 19 kB 00:00 2024-09-12T07:17:52.317 INFO:teuthology.orchestra.run.smithi064.stdout:(3/13): ceph-mgr-rook-19.3.0-4883.gd61b0886.el9 278 kB/s | 49 kB 00:00 2024-09-12T07:17:52.440 INFO:teuthology.orchestra.run.smithi064.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-09-12T07:17:52.473 INFO:teuthology.orchestra.run.smithi064.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.0 MB/s | 222 kB 00:00 2024-09-12T07:17:52.498 INFO:teuthology.orchestra.run.smithi064.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 914 kB/s | 54 kB 00:00 2024-09-12T07:17:52.532 INFO:teuthology.orchestra.run.smithi064.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.3 MB/s | 279 kB 00:00 2024-09-12T07:17:52.616 INFO:teuthology.orchestra.run.smithi064.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 226 kB/s | 32 kB 00:00 2024-09-12T07:17:52.641 INFO:teuthology.orchestra.run.smithi064.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 104 kB/s | 14 kB 00:00 2024-09-12T07:17:52.666 INFO:teuthology.orchestra.run.smithi064.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 1.7 MB/s | 220 kB 00:00 2024-09-12T07:17:52.725 INFO:teuthology.orchestra.run.smithi064.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 9.4 MB/s | 1.0 MB 00:00 2024-09-12T07:17:52.750 INFO:teuthology.orchestra.run.smithi064.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 712 kB/s | 59 kB 00:00 2024-09-12T07:17:52.775 INFO:teuthology.orchestra.run.smithi064.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 830 kB/s | 90 kB 00:00 2024-09-12T07:17:52.778 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:52.778 INFO:teuthology.orchestra.run.smithi064.stdout:Total 2.5 MB/s | 2.2 MB 00:00 2024-09-12T07:17:52.841 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4883.gd61b0 12/12 2024-09-12T07:17:52.842 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4883.gd61b0 1/12 2024-09-12T07:17:52.842 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-12T07:17:52.842 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-12T07:17:52.842 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-12T07:17:52.842 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-12T07:17:52.842 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-12T07:17:52.842 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-12T07:17:52.842 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-12T07:17:52.842 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-12T07:17:52.842 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-12T07:17:52.842 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-12T07:17:52.843 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:17:52.859 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:17:52.859 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:17:52.957 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:17:52.958 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:17:53.385 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-12T07:17:53.385 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:53.385 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:17:53.385 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:53.385 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T07:17:53.386 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T07:17:53.386 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T07:17:53.386 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T07:17:53.386 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T07:17:53.386 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T07:17:53.386 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T07:17:53.386 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T07:17:53.386 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T07:17:53.386 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T07:17:53.386 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T07:17:53.386 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:53.386 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:17:53.480 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:17:53.665 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-12T07:17:53.729 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-12T07:17:53.795 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-12T07:17:53.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-rook 2024-09-12T07:17:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-12T07:17:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-12T07:17:53.994 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-12T07:17:54.114 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-12T07:17:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-12T07:17:54.416 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:33 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:17:54.556 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:17:54.557 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-12T07:17:54.557 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:54.557 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:17:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 49 k 2024-09-12T07:17:54.557 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-12T07:17:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T07:17:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T07:17:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:17:54.558 INFO:teuthology.orchestra.run.smithi046.stdout:Install 13 Packages 2024-09-12T07:17:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:54.559 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 2.2 M 2024-09-12T07:17:54.559 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 26 M 2024-09-12T07:17:54.559 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:17:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-12T07:17:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-12T07:17:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-12T07:17:55.044 INFO:teuthology.orchestra.run.smithi046.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 253 kB/s | 26 kB 00:00 2024-09-12T07:17:55.069 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-12T07:17:55.070 INFO:teuthology.orchestra.run.smithi046.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 151 kB/s | 19 kB 00:00 2024-09-12T07:17:55.097 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:19.3.0-4883.gd61b0886.el9.noarch 13/13 2024-09-12T07:17:55.129 INFO:teuthology.orchestra.run.smithi046.stdout:(3/13): ceph-mgr-rook-19.3.0-4883.gd61b0886.el9 266 kB/s | 49 kB 00:00 2024-09-12T07:17:55.154 INFO:teuthology.orchestra.run.smithi046.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-12T07:17:55.179 INFO:teuthology.orchestra.run.smithi046.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-12T07:17:55.221 INFO:teuthology.orchestra.run.smithi046.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 804 kB/s | 54 kB 00:00 2024-09-12T07:17:55.288 INFO:teuthology.orchestra.run.smithi046.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.7 MB/s | 279 kB 00:00 2024-09-12T07:17:55.330 INFO:teuthology.orchestra.run.smithi046.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 214 kB/s | 32 kB 00:00 2024-09-12T07:17:55.355 INFO:teuthology.orchestra.run.smithi046.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 105 kB/s | 14 kB 00:00 2024-09-12T07:17:55.472 INFO:teuthology.orchestra.run.smithi046.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 507 kB/s | 59 kB 00:00 2024-09-12T07:17:55.539 INFO:teuthology.orchestra.run.smithi046.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 878 kB/s | 220 kB 00:00 2024-09-12T07:17:55.564 INFO:teuthology.orchestra.run.smithi046.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 976 kB/s | 90 kB 00:00 2024-09-12T07:17:55.614 INFO:teuthology.orchestra.run.smithi046.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.6 MB/s | 1.0 MB 00:00 2024-09-12T07:17:55.615 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:55.615 INFO:teuthology.orchestra.run.smithi046.stdout:Total 2.1 MB/s | 2.2 MB 00:01 2024-09-12T07:17:55.688 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:17:55.704 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:17:55.704 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:17:55.804 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:17:55.805 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:17:55.818 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4883.gd61b0886.el9.noarch 13/13 2024-09-12T07:17:55.818 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4883.gd61b0886.el9.noarch 1/13 2024-09-12T07:17:55.818 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-12T07:17:55.818 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-12T07:17:55.818 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-12T07:17:55.819 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-12T07:17:55.819 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-12T07:17:55.819 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-12T07:17:55.819 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-12T07:17:55.819 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-12T07:17:55.819 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-12T07:17:55.819 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-12T07:17:55.819 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-12T07:17:56.166 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-12T07:17:56.166 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:56.166 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:17:56.166 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:56.166 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T07:17:56.166 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T07:17:56.166 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T07:17:56.166 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T07:17:56.166 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T07:17:56.167 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T07:17:56.167 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T07:17:56.167 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T07:17:56.167 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T07:17:56.167 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T07:17:56.167 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T07:17:56.167 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T07:17:56.167 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:56.167 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:17:56.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-cephadm 2024-09-12T07:17:56.453 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:17:56.628 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-12T07:17:56.684 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-12T07:17:56.740 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-12T07:17:56.776 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-12T07:17:56.821 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-12T07:17:56.933 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-12T07:17:57.018 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:32 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:17:57.157 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:17:57.158 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:57.158 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-12T07:17:57.158 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:57.158 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:17:57.158 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 154 k 2024-09-12T07:17:57.158 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-12T07:17:57.158 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T07:17:57.158 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T07:17:57.158 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T07:17:57.159 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T07:17:57.159 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T07:17:57.159 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:57.159 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:17:57.159 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:17:57.159 INFO:teuthology.orchestra.run.smithi064.stdout:Install 6 Packages 2024-09-12T07:17:57.159 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:17:57.159 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 7.0 M 2024-09-12T07:17:57.160 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 33 M 2024-09-12T07:17:57.160 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:17:57.170 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-12T07:17:57.228 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-12T07:17:57.461 INFO:teuthology.orchestra.run.smithi064.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.8 MB/s | 248 kB 00:00 2024-09-12T07:17:57.545 INFO:teuthology.orchestra.run.smithi064.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4883.gd61b0886.e 700 kB/s | 154 kB 00:00 2024-09-12T07:17:57.628 INFO:teuthology.orchestra.run.smithi064.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-09-12T07:17:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-12T07:17:58.020 INFO:teuthology.orchestra.run.smithi064.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 121 kB/s | 58 kB 00:00 2024-09-12T07:17:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-12T07:17:58.045 INFO:teuthology.orchestra.run.smithi064.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 179 kB/s | 74 kB 00:00 2024-09-12T07:17:58.075 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-12T07:17:58.124 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-12T07:17:58.153 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:19.3.0-4883.gd61b0886.el9.noarch 13/13 2024-09-12T07:17:58.404 INFO:teuthology.orchestra.run.smithi064.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 582 kB/s | 548 kB 00:00 2024-09-12T07:17:58.405 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:17:58.405 INFO:teuthology.orchestra.run.smithi064.stdout:Total 5.6 MB/s | 7.0 MB 00:01 2024-09-12T07:17:58.481 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:17:58.494 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:17:58.494 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:17:58.600 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:17:58.601 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:17:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:17:58.941 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4883.gd61b0886.el9.noarch 13/13 2024-09-12T07:17:58.942 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4883.gd61b0886.el9.noarch 1/13 2024-09-12T07:17:58.942 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-12T07:17:58.942 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-12T07:17:58.942 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-12T07:17:58.942 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-12T07:17:58.942 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-12T07:17:58.942 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-12T07:17:58.942 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-12T07:17:58.942 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-12T07:17:58.942 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-12T07:17:58.942 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-12T07:17:58.942 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-12T07:17:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-12T07:17:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-12T07:17:59.299 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-12T07:17:59.299 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:59.299 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:17:59.299 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:17:59.299 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T07:17:59.299 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T07:17:59.299 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T07:17:59.300 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T07:17:59.300 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T07:17:59.300 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T07:17:59.300 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T07:17:59.300 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T07:17:59.300 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T07:17:59.300 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T07:17:59.300 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T07:17:59.300 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T07:17:59.300 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:17:59.300 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:17:59.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-cephadm 2024-09-12T07:17:59.727 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-12T07:17:59.812 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-12T07:17:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-12T07:17:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 6/6 2024-09-12T07:18:00.151 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:39 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:00.290 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:00.290 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:00.290 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 154 k 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout:Install 6 Packages 2024-09-12T07:18:00.291 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:00.292 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 7.0 M 2024-09-12T07:18:00.292 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 33 M 2024-09-12T07:18:00.292 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:18:00.689 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 6/6 2024-09-12T07:18:00.689 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 1/6 2024-09-12T07:18:00.689 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-12T07:18:00.689 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-12T07:18:00.689 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-12T07:18:00.689 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-12T07:18:00.951 INFO:teuthology.orchestra.run.smithi046.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4883.gd61b0886.e 788 kB/s | 154 kB 00:00 2024-09-12T07:18:00.973 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-12T07:18:00.973 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:00.973 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:18:00.973 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:18:00.974 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T07:18:00.974 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T07:18:00.974 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T07:18:00.974 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T07:18:00.974 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T07:18:00.974 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:00.974 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:01.018 INFO:teuthology.orchestra.run.smithi046.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 945 kB/s | 248 kB 00:00 2024-09-12T07:18:01.102 INFO:teuthology.orchestra.run.smithi046.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 694 kB/s | 58 kB 00:00 2024-09-12T07:18:01.127 INFO:teuthology.orchestra.run.smithi046.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.1 MB/s | 548 kB 00:00 2024-09-12T07:18:01.160 INFO:teuthology.orchestra.run.smithi046.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.2 MB/s | 74 kB 00:00 2024-09-12T07:18:01.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-fuse 2024-09-12T07:18:01.804 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:36 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:01.945 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-12T07:18:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:18:01.945 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 900 k 2024-09-12T07:18:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-12T07:18:01.945 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T07:18:01.945 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:18:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:01.945 INFO:teuthology.orchestra.run.smithi064.stdout:Install 2 Packages 2024-09-12T07:18:01.945 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 980 k 2024-09-12T07:18:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 2.9 M 2024-09-12T07:18:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:18:02.140 INFO:teuthology.orchestra.run.smithi064.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 463 kB/s | 80 kB 00:00 2024-09-12T07:18:02.215 INFO:teuthology.orchestra.run.smithi064.stdout:(2/2): ceph-fuse-19.3.0-4883.gd61b0886.el9.x86_ 3.6 MB/s | 900 kB 00:00 2024-09-12T07:18:02.215 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:02.215 INFO:teuthology.orchestra.run.smithi064.stdout:Total 3.5 MB/s | 980 kB 00:00 2024-09-12T07:18:02.228 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:18:02.250 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:18:02.250 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:18:02.337 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:18:02.337 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:18:02.532 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:18:02.680 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-12T07:18:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 2/2 2024-09-12T07:18:03.194 INFO:teuthology.orchestra.run.smithi046.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.4 MB/s | 6.0 MB 00:02 2024-09-12T07:18:03.195 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:03.195 INFO:teuthology.orchestra.run.smithi046.stdout:Total 2.4 MB/s | 7.0 MB 00:02 2024-09-12T07:18:03.271 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:18:03.283 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:18:03.283 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:18:03.390 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:18:03.390 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:18:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 2/2 2024-09-12T07:18:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 1/2 2024-09-12T07:18:03.725 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:18:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-12T07:18:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:03.789 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:18:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-12T07:18:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:03.789 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:03.917 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-12T07:18:03.981 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-12T07:18:04.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-volume 2024-09-12T07:18:04.538 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-12T07:18:04.627 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-12T07:18:04.692 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:39 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-12T07:18:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 6/6 2024-09-12T07:18:04.833 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:04.834 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:04.834 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-12T07:18:04.834 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:04.834 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:18:04.834 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 285 k 2024-09-12T07:18:04.834 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:04.834 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:18:04.834 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:04.834 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-09-12T07:18:04.834 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:04.834 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 285 k 2024-09-12T07:18:04.834 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 1.3 M 2024-09-12T07:18:04.835 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:18:05.045 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-volume-19.3.0-4883.gd61b0886.el9.noarch.rp 1.3 MB/s | 285 kB 00:00 2024-09-12T07:18:05.046 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:05.046 INFO:teuthology.orchestra.run.smithi064.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-12T07:18:05.046 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:18:05.055 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:18:05.055 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:18:05.082 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:18:05.082 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:18:05.317 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:18:05.389 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:18:05.405 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:18:05.406 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:18:05.406 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T07:18:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:05.516 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 6/6 2024-09-12T07:18:05.517 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 1/6 2024-09-12T07:18:05.517 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-12T07:18:05.517 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-12T07:18:05.517 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-12T07:18:05.517 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-12T07:18:05.806 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-12T07:18:05.806 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:05.806 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:18:05.806 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:18:05.806 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T07:18:05.806 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T07:18:05.806 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T07:18:05.806 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T07:18:05.806 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T07:18:05.807 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:06.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-fuse 2024-09-12T07:18:06.451 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:18:06.451 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:06.451 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:18:06.451 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:18:06.452 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:06.452 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:06.628 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:45 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:06.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librados-devel 2024-09-12T07:18:06.770 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:06.770 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:06.770 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-12T07:18:06.770 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:06.770 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:18:06.770 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 900 k 2024-09-12T07:18:06.771 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-12T07:18:06.771 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T07:18:06.771 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:06.771 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:18:06.771 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:06.771 INFO:teuthology.orchestra.run.smithi046.stdout:Install 2 Packages 2024-09-12T07:18:06.771 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:06.771 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 980 k 2024-09-12T07:18:06.772 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 2.9 M 2024-09-12T07:18:06.772 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:18:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 529 kB/s | 80 kB 00:00 2024-09-12T07:18:07.356 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:42 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:07.363 INFO:teuthology.orchestra.run.smithi046.stdout:(2/2): ceph-fuse-19.3.0-4883.gd61b0886.el9.x86_ 1.8 MB/s | 900 kB 00:00 2024-09-12T07:18:07.364 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:07.364 INFO:teuthology.orchestra.run.smithi046.stdout:Total 1.6 MB/s | 980 kB 00:00 2024-09-12T07:18:07.376 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:18:07.399 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:18:07.400 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:18:07.486 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:18:07.486 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:18:07.495 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:07.496 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:07.497 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-09-12T07:18:07.497 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:07.497 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:18:07.497 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 127 k 2024-09-12T07:18:07.497 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:07.497 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:18:07.497 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:07.497 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-09-12T07:18:07.497 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:07.497 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 127 k 2024-09-12T07:18:07.498 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 456 k 2024-09-12T07:18:07.498 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:18:07.681 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:18:07.709 INFO:teuthology.orchestra.run.smithi064.stdout:librados-devel-19.3.0-4883.gd61b0886.el9.x86_64 599 kB/s | 127 kB 00:00 2024-09-12T07:18:07.709 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:07.709 INFO:teuthology.orchestra.run.smithi064.stdout:Total 595 kB/s | 127 kB 00:00 2024-09-12T07:18:07.710 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:18:07.715 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:18:07.716 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:18:07.784 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:18:07.784 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:18:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-12T07:18:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 2/2 2024-09-12T07:18:07.942 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:18:08.059 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:08.502 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:08.609 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 2/2 2024-09-12T07:18:08.609 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 1/2 2024-09-12T07:18:08.759 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:08.759 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:08.760 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:18:08.760 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:18:08.760 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:08.760 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:08.935 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-12T07:18:08.935 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:08.935 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:18:08.935 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-12T07:18:08.935 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:08.936 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:08.994 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install libcephfs2 2024-09-12T07:18:09.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-volume 2024-09-12T07:18:09.598 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:44 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:09.693 INFO:teuthology.orchestra.run.smithi064.stdout:Package libcephfs2-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:09.740 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:09.741 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-09-12T07:18:09.741 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:09.823 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install libcephfs-devel 2024-09-12T07:18:09.834 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:48 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:09.976 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-12T07:18:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:18:09.977 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume noarch 2:19.3.0-4883.gd61b0886.el9 ceph-noarch 285 k 2024-09-12T07:18:09.977 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:18:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-12T07:18:09.977 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 285 k 2024-09-12T07:18:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 1.3 M 2024-09-12T07:18:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:18:10.211 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-volume-19.3.0-4883.gd61b0886.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-12T07:18:10.211 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:10.212 INFO:teuthology.orchestra.run.smithi046.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-12T07:18:10.212 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:18:10.221 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:18:10.221 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:18:10.248 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:18:10.249 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:18:10.412 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:45 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:10.481 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:18:10.546 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-volume-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:18:10.552 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:10.553 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:10.553 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-09-12T07:18:10.553 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:10.553 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:18:10.553 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 32 k 2024-09-12T07:18:10.553 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:10.554 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:18:10.554 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:10.554 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-09-12T07:18:10.554 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:10.554 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 32 k 2024-09-12T07:18:10.554 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 142 k 2024-09-12T07:18:10.554 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:18:10.563 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-volume-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:18:10.563 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:18:10.563 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T07:18:10.563 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:10.729 INFO:teuthology.orchestra.run.smithi064.stdout:libcephfs-devel-19.3.0-4883.gd61b0886.el9.x86_6 181 kB/s | 32 kB 00:00 2024-09-12T07:18:10.729 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:10.730 INFO:teuthology.orchestra.run.smithi064.stdout:Total 180 kB/s | 32 kB 00:00 2024-09-12T07:18:10.730 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:18:10.733 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:18:10.734 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:18:10.757 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:18:10.855 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:18:10.855 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:18:10.964 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:11.349 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:11.547 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-volume-2:19.3.0-4883.gd61b0886.el9.noarch 1/1 2024-09-12T07:18:11.547 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:11.547 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:18:11.547 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume-2:19.3.0-4883.gd61b0886.el9.noarch 2024-09-12T07:18:11.547 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:11.548 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:11.597 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:11.597 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:11.597 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:18:11.597 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:18:11.597 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:11.597 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:11.786 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librados2 2024-09-12T07:18:11.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install librados-devel 2024-09-12T07:18:12.373 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:47 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:12.443 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:51 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:12.466 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:12.512 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:12.513 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-09-12T07:18:12.513 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:12.582 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:12.583 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-09-12T07:18:12.583 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:12.583 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:18:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 127 k 2024-09-12T07:18:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:12.583 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:18:12.584 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:12.584 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-12T07:18:12.584 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:12.584 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 127 k 2024-09-12T07:18:12.584 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 456 k 2024-09-12T07:18:12.584 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:18:12.586 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librbd1 2024-09-12T07:18:12.783 INFO:teuthology.orchestra.run.smithi046.stdout:librados-devel-19.3.0-4883.gd61b0886.el9.x86_64 638 kB/s | 127 kB 00:00 2024-09-12T07:18:12.783 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:12.784 INFO:teuthology.orchestra.run.smithi046.stdout:Total 633 kB/s | 127 kB 00:00 2024-09-12T07:18:12.784 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:18:12.790 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:18:12.790 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:18:12.859 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:18:12.860 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:18:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:18:13.134 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:13.178 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:48 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:13.271 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:13.316 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:13.317 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-09-12T07:18:13.317 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:13.395 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rados 2024-09-12T07:18:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:13.953 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:13.953 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:13.953 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:18:13.953 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:18:13.953 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:13.953 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:13.982 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:48 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:14.075 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rados-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:14.121 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:14.122 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-09-12T07:18:14.122 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:14.198 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rgw 2024-09-12T07:18:14.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install libcephfs2 2024-09-12T07:18:14.789 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:49 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:14.813 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:53 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:14.886 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rgw-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:14.906 INFO:teuthology.orchestra.run.smithi046.stdout:Package libcephfs2-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:14.932 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:14.933 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-09-12T07:18:14.933 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:14.952 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-12T07:18:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:15.014 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-cephfs 2024-09-12T07:18:15.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install libcephfs-devel 2024-09-12T07:18:15.603 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:50 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:15.609 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:54 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:15.697 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-cephfs-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:15.743 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:15.744 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-09-12T07:18:15.744 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:15.748 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:15.749 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-09-12T07:18:15.749 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:15.749 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:18:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 32 k 2024-09-12T07:18:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:15.749 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:18:15.749 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:15.750 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-12T07:18:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:15.750 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 32 k 2024-09-12T07:18:15.750 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 142 k 2024-09-12T07:18:15.750 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:18:15.873 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rbd 2024-09-12T07:18:15.904 INFO:teuthology.orchestra.run.smithi046.stdout:libcephfs-devel-19.3.0-4883.gd61b0886.el9.x86_6 206 kB/s | 32 kB 00:00 2024-09-12T07:18:15.904 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:15.905 INFO:teuthology.orchestra.run.smithi046.stdout:Total 204 kB/s | 32 kB 00:00 2024-09-12T07:18:15.905 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:18:15.909 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:18:15.909 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:18:15.923 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:18:15.924 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:18:16.016 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:18:16.138 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:16.467 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:51 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:16.504 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:16.561 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rbd-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:16.607 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:16.608 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-09-12T07:18:16.608 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:16.690 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-fuse 2024-09-12T07:18:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:16.754 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:16.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install librados2 2024-09-12T07:18:17.274 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:52 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:17.412 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:17.413 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:17.413 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-09-12T07:18:17.413 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:17.413 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:18:17.413 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 85 k 2024-09-12T07:18:17.413 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:17.414 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:18:17.414 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:17.414 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-09-12T07:18:17.414 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:17.414 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 85 k 2024-09-12T07:18:17.414 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 231 k 2024-09-12T07:18:17.414 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:18:17.529 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:56 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:17.616 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-fuse-19.3.0-4883.gd61b0886.el9.x86_64.rpm 422 kB/s | 85 kB 00:00 2024-09-12T07:18:17.616 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:17.617 INFO:teuthology.orchestra.run.smithi064.stdout:Total 419 kB/s | 85 kB 00:00 2024-09-12T07:18:17.617 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:18:17.621 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:17.624 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:18:17.624 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:18:17.667 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:17.668 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-12T07:18:17.669 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:17.692 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:18:17.692 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:18:17.732 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install librbd1 2024-09-12T07:18:17.951 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:18:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:18.321 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:57 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:18.415 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:18.427 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:18.460 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:18.462 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-12T07:18:18.462 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:18.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-rados 2024-09-12T07:18:18.714 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:18.714 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:18.715 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:18:18.715 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:18:18.715 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:18.715 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:18.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-mirror 2024-09-12T07:18:19.140 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:58 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:19.232 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-rados-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:19.278 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:19.279 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-12T07:18:19.279 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:19.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-rgw 2024-09-12T07:18:19.594 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:54 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:19.737 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:19.738 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:19.738 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-09-12T07:18:19.738 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:19.738 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:18:19.738 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 3.2 M 2024-09-12T07:18:19.738 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:19.738 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:18:19.738 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:19.738 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-09-12T07:18:19.738 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:19.738 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 3.2 M 2024-09-12T07:18:19.738 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 14 M 2024-09-12T07:18:19.739 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:18:19.940 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:58 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:20.035 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-rgw-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:20.070 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-mirror-19.3.0-4883.gd61b0886.el9.x86_64.rpm 9.6 MB/s | 3.2 MB 00:00 2024-09-12T07:18:20.071 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:20.071 INFO:teuthology.orchestra.run.smithi064.stdout:Total 9.6 MB/s | 3.2 MB 00:00 2024-09-12T07:18:20.072 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:18:20.081 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:18:20.082 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:18:20.082 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:20.083 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-12T07:18:20.083 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:20.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-cephfs 2024-09-12T07:18:20.168 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:18:20.169 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:18:20.632 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:18:20.705 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:20.733 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:20.733 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:18:20.733 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T07:18:20.733 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T07:18:20.733 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T07:18:20.733 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:20.753 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:59 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:20.850 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-cephfs-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:20.896 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:20.897 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-12T07:18:20.897 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:20.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-rbd 2024-09-12T07:18:21.584 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:00 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:21.679 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-rbd-2:19.3.0-4883.gd61b0886.el9.x86_64 is already installed. 2024-09-12T07:18:21.726 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:21.727 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-12T07:18:21.727 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:21.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install rbd-fuse 2024-09-12T07:18:21.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:21.922 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:21.922 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:18:21.922 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:18:21.922 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:21.922 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:22.173 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-nbd 2024-09-12T07:18:22.379 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:01 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:22.519 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:22.519 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-09-12T07:18:22.520 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:22.520 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:18:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 85 k 2024-09-12T07:18:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:22.520 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:18:22.520 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:22.520 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-12T07:18:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:22.520 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 85 k 2024-09-12T07:18:22.520 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 231 k 2024-09-12T07:18:22.520 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:18:22.716 INFO:teuthology.orchestra.run.smithi046.stdout:rbd-fuse-19.3.0-4883.gd61b0886.el9.x86_64.rpm 436 kB/s | 85 kB 00:00 2024-09-12T07:18:22.716 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:22.717 INFO:teuthology.orchestra.run.smithi046.stdout:Total 433 kB/s | 85 kB 00:00 2024-09-12T07:18:22.717 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:18:22.724 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:18:22.724 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:18:22.772 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:57 ago on Thu 12 Sep 2024 07:16:25 AM UTC. 2024-09-12T07:18:22.792 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:18:22.793 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:18:22.916 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T07:18:22.918 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:22.920 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-09-12T07:18:22.921 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:22.922 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T07:18:22.924 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 171 k 2024-09-12T07:18:22.925 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:22.925 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T07:18:22.926 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T07:18:22.926 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-09-12T07:18:22.926 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:22.926 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 171 k 2024-09-12T07:18:22.926 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 490 k 2024-09-12T07:18:22.926 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T07:18:22.948 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:18:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:23.125 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-nbd-19.3.0-4883.gd61b0886.el9.x86_64.rpm 808 kB/s | 171 kB 00:00 2024-09-12T07:18:23.126 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:23.126 INFO:teuthology.orchestra.run.smithi064.stdout:Total 802 kB/s | 171 kB 00:00 2024-09-12T07:18:23.126 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T07:18:23.135 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T07:18:23.135 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T07:18:23.204 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T07:18:23.204 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T07:18:23.360 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T07:18:23.460 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:23.744 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:23.745 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:18:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:18:23.745 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:23.745 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:23.921 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:24.011 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install rbd-mirror 2024-09-12T07:18:24.187 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:24.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:24.187 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T07:18:24.187 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:18:24.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:24.187 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T07:18:24.438 DEBUG:teuthology.parallel:result is None 2024-09-12T07:18:24.612 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:03 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:24.752 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:24.753 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:24.753 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-09-12T07:18:24.753 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:24.753 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:18:24.753 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 3.2 M 2024-09-12T07:18:24.753 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:24.753 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:18:24.753 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:24.753 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-12T07:18:24.754 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:24.754 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 3.2 M 2024-09-12T07:18:24.754 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 14 M 2024-09-12T07:18:24.754 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:18:25.086 INFO:teuthology.orchestra.run.smithi046.stdout:rbd-mirror-19.3.0-4883.gd61b0886.el9.x86_64.rpm 9.6 MB/s | 3.2 MB 00:00 2024-09-12T07:18:25.087 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:25.087 INFO:teuthology.orchestra.run.smithi046.stdout:Total 9.6 MB/s | 3.2 MB 00:00 2024-09-12T07:18:25.087 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:18:25.097 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:18:25.098 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:18:25.184 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:18:25.185 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:18:25.654 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:18:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:25.730 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T07:18:25.730 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T07:18:25.731 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T07:18:25.731 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T07:18:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:26.911 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:18:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:18:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:26.911 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:27.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install rbd-nbd 2024-09-12T07:18:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:06 ago on Thu 12 Sep 2024 07:16:21 AM UTC. 2024-09-12T07:18:27.867 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T07:18:27.868 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:27.868 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-09-12T07:18:27.868 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:27.868 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T07:18:27.868 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:19.3.0-4883.gd61b0886.el9 ceph 171 k 2024-09-12T07:18:27.868 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:27.868 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T07:18:27.868 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T07:18:27.868 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-12T07:18:27.869 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:27.869 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 171 k 2024-09-12T07:18:27.869 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 490 k 2024-09-12T07:18:27.869 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T07:18:28.048 INFO:teuthology.orchestra.run.smithi046.stdout:rbd-nbd-19.3.0-4883.gd61b0886.el9.x86_64.rpm 953 kB/s | 171 kB 00:00 2024-09-12T07:18:28.049 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T07:18:28.049 INFO:teuthology.orchestra.run.smithi046.stdout:Total 945 kB/s | 171 kB 00:00 2024-09-12T07:18:28.049 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T07:18:28.058 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T07:18:28.058 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T07:18:28.125 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T07:18:28.126 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T07:18:28.292 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T07:18:28.408 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:29.243 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:19.3.0-4883.gd61b0886.el9.x86_64 1/1 2024-09-12T07:18:29.243 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:29.243 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T07:18:29.243 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:19.3.0-4883.gd61b0886.el9.x86_64 2024-09-12T07:18:29.244 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:29.244 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T07:18:29.461 DEBUG:teuthology.parallel:result is None 2024-09-12T07:18:29.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d61b08865d4d98738f45ff2e4806d7fb6d03eca1 2024-09-12T07:18:29.613 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T07:18:29.636 INFO:teuthology.orchestra.run.smithi046.stdout:19.3.0-4883.gd61b0886.el9 2024-09-12T07:18:29.636 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4883.gd61b0886.el9 2024-09-12T07:18:29.636 INFO:teuthology.task.install:The correct ceph version 19.3.0-4883.gd61b0886 is installed. 2024-09-12T07:18:29.638 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d61b08865d4d98738f45ff2e4806d7fb6d03eca1 2024-09-12T07:18:29.778 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T07:18:29.802 INFO:teuthology.orchestra.run.smithi064.stdout:19.3.0-4883.gd61b0886.el9 2024-09-12T07:18:29.802 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4883.gd61b0886.el9 2024-09-12T07:18:29.803 INFO:teuthology.task.install:The correct ceph version 19.3.0-4883.gd61b0886 is installed. 2024-09-12T07:18:29.804 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T07:18:29.805 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:18:29.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T07:18:29.833 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:18:29.833 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T07:18:29.871 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T07:18:29.871 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:18:29.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T07:18:29.901 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T07:18:29.967 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:18:29.967 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T07:18:29.996 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T07:18:30.065 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T07:18:30.065 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:18:30.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T07:18:30.093 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T07:18:30.160 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:18:30.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T07:18:30.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T07:18:30.257 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T07:18:30.258 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:18:30.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T07:18:30.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T07:18:30.352 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:18:30.352 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T07:18:30.382 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T07:18:30.449 INFO:teuthology.run_tasks:Running task ceph... 2024-09-12T07:18:30.529 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-12T07:18:30.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /var/log/ceph 2024-09-12T07:18:30.532 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /var/log/ceph 2024-09-12T07:18:30.558 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-12T07:18:30.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-12T07:18:30.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-12T07:18:30.624 INFO:tasks.ceph:Creating extra log directories... 2024-09-12T07:18:30.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-12T07:18:30.666 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-12T07:18:30.700 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-12T07:18:30.701 INFO:tasks.ceph:config {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-12T07:18:30.701 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413', 'branch': 'wip-dis-testing', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit}', 'email': 'idryomov@gmail.com', 'first_in_suite': False, 'job_id': '7902413', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing', 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}}, 'owner': 'scheduled_dis@teuthology', 'priority': 70, 'repo': 'https://github.com/ceph/ceph-ci', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1680, 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1', 'sleep_before_teardown': 0, 'subset': '42/100', 'suite': 'rbd', 'suite_branch': 'wip-dis-testing', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d61b08865d4d98738f45ff2e4806d7fb6d03eca1/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci', 'suite_sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1', 'targets': {'smithi046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAcfZXe0VRHaEtk5S4R7vw7f/noGB1DFAB0c4pyLi6se0ro/S2JD050NXdiPBgqRPFJhugEW+viUsiyTy7zBj3I=', 'smithi064.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPaE0wZSgmsbTYV6ikm6nO63TyJAIm/m0qwh5PQRHOzPocDz4O+fVtn0ERmZrGM1026FtAtrBKLqu8anICcxTBY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-12_04:46:10', 'tube': 'smithi', 'user': 'dis', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-12T07:18:30.701 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-12T07:18:30.733 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-12T07:18:30.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T07:18:30.790 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T07:18:30.815 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:18:30.816 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T07:18:30.872 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T07:18:30.872 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-09-12T07:18:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T07:18:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T07:18:30.928 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-12T07:18:30.928 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T07:18:30.928 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T07:18:30.928 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-12 07:18:25.984589247 +0000 2024-09-12T07:18:30.928 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-12 07:15:44.446871170 +0000 2024-09-12T07:18:30.928 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-12 07:15:44.446871170 +0000 2024-09-12T07:18:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-12 07:15:44.446871170 +0000 2024-09-12T07:18:30.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T07:18:30.992 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-12T07:18:30.993 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-12T07:18:30.993 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 8.9611e-05 s, 5.7 MB/s 2024-09-12T07:18:30.994 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T07:18:31.052 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-09-12T07:18:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T07:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T07:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-12T07:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T07:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T07:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-12 07:18:25.985589236 +0000 2024-09-12T07:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-12 07:15:44.774867809 +0000 2024-09-12T07:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-12 07:15:44.774867809 +0000 2024-09-12T07:18:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-12 07:15:44.774867809 +0000 2024-09-12T07:18:31.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T07:18:31.173 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-12T07:18:31.173 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-12T07:18:31.173 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000104425 s, 4.9 MB/s 2024-09-12T07:18:31.174 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T07:18:31.231 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-09-12T07:18:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T07:18:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T07:18:31.288 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T07:18:31.288 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T07:18:31.289 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T07:18:31.289 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-12 07:18:25.985589236 +0000 2024-09-12T07:18:31.289 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-12 07:15:45.105864417 +0000 2024-09-12T07:18:31.289 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-12 07:15:45.105864417 +0000 2024-09-12T07:18:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-12 07:15:45.105864417 +0000 2024-09-12T07:18:31.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T07:18:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-12T07:18:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-12T07:18:31.353 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000100811 s, 5.1 MB/s 2024-09-12T07:18:31.354 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T07:18:31.411 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-09-12T07:18:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T07:18:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T07:18:31.468 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-12T07:18:31.468 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T07:18:31.468 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T07:18:31.469 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-12 07:18:25.985589236 +0000 2024-09-12T07:18:31.469 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-12 07:15:45.426861128 +0000 2024-09-12T07:18:31.469 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-12 07:15:45.426861128 +0000 2024-09-12T07:18:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-12 07:15:45.426861128 +0000 2024-09-12T07:18:31.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T07:18:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-12T07:18:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-12T07:18:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.00010137 s, 5.1 MB/s 2024-09-12T07:18:31.534 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T07:18:31.591 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T07:18:31.591 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:18:31.591 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T07:18:31.611 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T07:18:31.611 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-09-12T07:18:31.670 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T07:18:31.670 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T07:18:31.671 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-12T07:18:31.671 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T07:18:31.671 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T07:18:31.671 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-12 07:18:20.997407040 +0000 2024-09-12T07:18:31.671 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-12 07:15:42.430312057 +0000 2024-09-12T07:18:31.671 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-12 07:15:42.430312057 +0000 2024-09-12T07:18:31.671 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-12 07:15:42.430312057 +0000 2024-09-12T07:18:31.671 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T07:18:31.739 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-12T07:18:31.739 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-12T07:18:31.739 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000206705 s, 2.5 MB/s 2024-09-12T07:18:31.741 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T07:18:31.803 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-09-12T07:18:31.861 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T07:18:31.862 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T07:18:31.862 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-12T07:18:31.862 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T07:18:31.862 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T07:18:31.862 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-12 07:18:20.997407040 +0000 2024-09-12T07:18:31.862 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-12 07:15:42.769304861 +0000 2024-09-12T07:18:31.862 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-12 07:15:42.769304861 +0000 2024-09-12T07:18:31.862 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-12 07:15:42.769304861 +0000 2024-09-12T07:18:31.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T07:18:31.928 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-12T07:18:31.928 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-12T07:18:31.928 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000202863 s, 2.5 MB/s 2024-09-12T07:18:31.929 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T07:18:31.987 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-09-12T07:18:32.044 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T07:18:32.045 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T07:18:32.045 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-12T07:18:32.045 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T07:18:32.045 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T07:18:32.045 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-12 07:18:20.998407020 +0000 2024-09-12T07:18:32.045 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-12 07:15:43.115297517 +0000 2024-09-12T07:18:32.045 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-12 07:15:43.115297517 +0000 2024-09-12T07:18:32.045 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-12 07:15:43.115297517 +0000 2024-09-12T07:18:32.045 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T07:18:32.110 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-12T07:18:32.110 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-12T07:18:32.110 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000217027 s, 2.4 MB/s 2024-09-12T07:18:32.111 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T07:18:32.169 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-09-12T07:18:32.226 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T07:18:32.226 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T07:18:32.226 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-12T07:18:32.226 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T07:18:32.226 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T07:18:32.227 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-12 07:18:20.998407020 +0000 2024-09-12T07:18:32.227 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-12 07:15:43.445290512 +0000 2024-09-12T07:18:32.227 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-12 07:15:43.445290512 +0000 2024-09-12T07:18:32.227 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-12 07:15:43.445290512 +0000 2024-09-12T07:18:32.227 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T07:18:32.292 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-12T07:18:32.292 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-12T07:18:32.292 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000224934 s, 2.3 MB/s 2024-09-12T07:18:32.293 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T07:18:32.351 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T07:18:32.351 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-12T07:18:32.351 INFO:tasks.ceph:Generating config... 2024-09-12T07:18:32.352 INFO:tasks.ceph:[client] rbd qos iops burst = 100 2024-09-12T07:18:32.352 INFO:tasks.ceph:[client] rbd qos iops limit = 50 2024-09-12T07:18:32.353 INFO:tasks.ceph:[client] rbd qos schedule tick min = 100 2024-09-12T07:18:32.353 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-12T07:18:32.353 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-12T07:18:32.353 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-12T07:18:32.353 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-12T07:18:32.353 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-12T07:18:32.353 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-12T07:18:32.353 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-12T07:18:32.353 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-12T07:18:32.353 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-12T07:18:32.353 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-12T07:18:32.353 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-12T07:18:32.353 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-12T07:18:32.353 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-12T07:18:32.354 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-12T07:18:32.355 INFO:tasks.ceph:Setting up mon.a... 2024-09-12T07:18:32.355 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-12T07:18:32.419 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.keyring 2024-09-12T07:18:32.422 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-12T07:18:32.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T07:18:32.494 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.46'), ('mon.c', '[v2:172.21.15.46:3301,v1:172.21.15.46:6790]'), ('mon.b', '172.21.15.64')] 2024-09-12T07:18:32.495 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.46,[v2:172.21.15.46:3301,v1:172.21.15.46:6790],172.21.15.64', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-12T07:18:32.496 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:18:32.496 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T07:18:32.550 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.46 --addv c '[v2:172.21.15.46:3301,v1:172.21.15.46:6790]' --add b 172.21.15.64 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:18:32.638 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:18:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: generated fsid 8412ed58-ac5e-4549-b523-1a58cf9f9429 2024-09-12T07:18:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = quincy 2024-09-12T07:18:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 0 2024-09-12T07:18:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:fsid 8412ed58-ac5e-4549-b523-1a58cf9f9429 2024-09-12T07:18:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:last_changed 2024-09-12T07:18:32.640881+0000 2024-09-12T07:18:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-12T07:18:32.640881+0000 2024-09-12T07:18:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:min_mon_release 17 (quincy) 2024-09-12T07:18:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:election_strategy: 1 2024-09-12T07:18:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-12T07:18:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.b 2024-09-12T07:18:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:2: [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] mon.c 2024-09-12T07:18:32.639 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-12T07:18:32.641 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T07:18:32.657 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8412ed58-ac5e-4549-b523-1a58cf9f9429... 2024-09-12T07:18:32.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T07:18:32.700 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T07:18:32.741 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-09-12T07:18:32.741 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-09-12T07:18:32.741 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T07:18:32.741 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-09-12T07:18:32.741 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.741 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-09-12T07:18:32.741 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.742 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-09-12T07:18:32.742 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.742 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-09-12T07:18:32.742 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-09-12T07:18:32.742 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.742 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-09-12T07:18:32.742 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-09-12T07:18:32.742 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.742 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T07:18:32.742 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-09-12T07:18:32.742 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.742 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T07:18:32.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-09-12T07:18:32.744 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.744 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T07:18:32.744 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.744 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-09-12T07:18:32.744 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.744 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,[v2:172.21.15.46:3301,v1:172.21.15.46:6790],172.21.15.64" 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout: mon client directed command retry = 5 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on pool no app = False 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout: ms inject socket failures = 5000 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 8412ed58-ac5e-4549-b523-1a58cf9f9429 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-09-12T07:18:32.745 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.746 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-09-12T07:18:32.746 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-09-12T07:18:32.746 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-09-12T07:18:32.746 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.746 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-09-12T07:18:32.746 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-09-12T07:18:32.746 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.746 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-09-12T07:18:32.746 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-09-12T07:18:32.746 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-09-12T07:18:32.746 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-09-12T07:18:32.746 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.747 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-09-12T07:18:32.747 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-09-12T07:18:32.747 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.747 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-09-12T07:18:32.747 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.747 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-09-12T07:18:32.748 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.748 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-09-12T07:18:32.748 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.748 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-09-12T07:18:32.748 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-09-12T07:18:32.748 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.748 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-09-12T07:18:32.748 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-09-12T07:18:32.748 INFO:teuthology.orchestra.run.smithi046.stdout: bdev async discard = True 2024-09-12T07:18:32.748 INFO:teuthology.orchestra.run.smithi046.stdout: bdev enable discard = True 2024-09-12T07:18:32.748 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: osd blocked scrub grace period = 3600 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: osd max markdown count = 1000 2024-09-12T07:18:32.749 INFO:teuthology.orchestra.run.smithi046.stdout: osd memory target = 1610612736 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout: osd shutdown pgref assert = True 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.750 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T07:18:32.751 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-09-12T07:18:32.752 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-09-12T07:18:32.753 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-09-12T07:18:32.753 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-09-12T07:18:32.753 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T07:18:32.753 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T07:18:32.753 INFO:teuthology.orchestra.run.smithi046.stdout: rbd qos iops burst = 100 2024-09-12T07:18:32.753 INFO:teuthology.orchestra.run.smithi046.stdout: rbd qos iops limit = 50 2024-09-12T07:18:32.753 INFO:teuthology.orchestra.run.smithi046.stdout: rbd qos schedule tick min = 100 2024-09-12T07:18:32.753 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-09-12T07:18:32.753 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-09-12T07:18:32.753 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-09-12T07:18:32.754 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2024-09-12T07:18:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2024-09-12T07:18:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T07:18:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2024-09-12T07:18:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2024-09-12T07:18:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2024-09-12T07:18:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2024-09-12T07:18:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2024-09-12T07:18:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2024-09-12T07:18:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2024-09-12T07:18:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T07:18:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2024-09-12T07:18:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2024-09-12T07:18:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T07:18:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T07:18:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: mon_allow_pool_size_one = true 2024-09-12T07:18:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T07:18:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2024-09-12T07:18:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2024-09-12T07:18:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2024-09-12T07:18:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2024-09-12T07:18:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2024-09-12T07:18:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.46,[v2:172.21.15.46:3301,v1:172.21.15.46:6790],172.21.15.64" 2024-09-12T07:18:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: mon client directed command retry = 5 2024-09-12T07:18:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on pool no app = False 2024-09-12T07:18:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: ms inject socket failures = 5000 2024-09-12T07:18:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 8412ed58-ac5e-4549-b523-1a58cf9f9429 2024-09-12T07:18:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.758 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2024-09-12T07:18:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2024-09-12T07:18:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2024-09-12T07:18:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2024-09-12T07:18:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: osd mclock profile = high_recovery_ops 2024-09-12T07:18:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2024-09-12T07:18:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2024-09-12T07:18:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2024-09-12T07:18:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2024-09-12T07:18:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2024-09-12T07:18:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug trim objects = true 2024-09-12T07:18:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2024-09-12T07:18:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2024-09-12T07:18:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2024-09-12T07:18:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2024-09-12T07:18:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: bdev async discard = True 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: bdev enable discard = True 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 1/20 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 1/20 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-09-12T07:18:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 4/10 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: osd blocked scrub grace period = 3600 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: osd max markdown count = 1000 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: osd memory target = 1610612736 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: osd shutdown pgref assert = True 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-09-12T07:18:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2024-09-12T07:18:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: # 1m isn't quite enough 2024-09-12T07:18:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: mon_down_mkfs_grace = 2m 2024-09-12T07:18:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_filestore_osds = false 2024-09-12T07:18:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T07:18:32.765 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2024-09-12T07:18:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2024-09-12T07:18:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2024-09-12T07:18:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2024-09-12T07:18:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T07:18:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T07:18:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: rbd qos iops burst = 100 2024-09-12T07:18:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: rbd qos iops limit = 50 2024-09-12T07:18:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: rbd qos schedule tick min = 100 2024-09-12T07:18:32.766 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2024-09-12T07:18:32.766 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2024-09-12T07:18:32.766 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2024-09-12T07:18:32.767 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-12T07:18:32.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-12T07:18:32.864 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-12T07:18:32.864 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:18:32.864 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-12T07:18:32.881 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:18:32.881 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-12T07:18:32.938 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T07:18:32.938 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:18:32.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-12T07:18:32.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T07:18:33.014 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:18:33.014 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:18:33.070 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T07:18:33.071 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:18:33.071 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-12T07:18:33.071 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T07:18:33.114 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:18:33.114 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:18:33.172 INFO:tasks.ceph:Setting up mon nodes... 2024-09-12T07:18:33.173 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-12T07:18:33.173 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-12T07:18:33.225 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-12T07:18:33.228 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-12T07:18:33.290 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-12T07:18:33.293 INFO:tasks.ceph:Setting up mds nodes... 2024-09-12T07:18:33.293 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-12T07:18:33.293 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T07:18:33.337 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-12T07:18:33.350 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-12T07:18:33.397 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-12T07:18:33.410 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-12T07:18:33.410 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-12T07:18:33.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-12T07:18:33.435 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T07:18:33.435 INFO:tasks.ceph:role: osd.0 2024-09-12T07:18:33.436 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T07:18:33.436 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-12T07:18:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T07:18:33.500 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T07:18:33.500 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T07:18:33.500 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T07:18:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T07:18:33.501 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-12T07:18:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T07:18:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T07:18:33.501 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T07:18:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T07:18:33.532 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-09-12T07:18:33.534 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-12T07:18:33.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-12T07:18:33.605 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-12T07:18:33.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-12T07:18:33.740 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T07:18:33.740 INFO:tasks.ceph:role: osd.1 2024-09-12T07:18:33.741 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T07:18:33.741 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-12T07:18:33.805 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T07:18:33.805 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T07:18:33.806 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T07:18:33.806 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T07:18:33.806 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T07:18:33.806 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-12T07:18:33.806 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T07:18:33.806 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T07:18:33.806 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T07:18:33.806 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T07:18:33.837 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-09-12T07:18:33.839 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-12T07:18:33.839 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-12T07:18:33.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-12T07:18:33.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-12T07:18:34.044 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T07:18:34.044 INFO:tasks.ceph:role: osd.2 2024-09-12T07:18:34.044 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T07:18:34.044 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-12T07:18:34.109 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T07:18:34.109 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T07:18:34.109 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T07:18:34.109 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T07:18:34.109 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T07:18:34.109 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-12T07:18:34.110 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T07:18:34.110 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T07:18:34.110 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T07:18:34.110 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T07:18:34.142 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-09-12T07:18:34.144 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-12T07:18:34.145 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-12T07:18:34.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-12T07:18:34.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-12T07:18:34.349 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T07:18:34.349 INFO:tasks.ceph:role: osd.3 2024-09-12T07:18:34.349 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T07:18:34.349 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-12T07:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T07:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T07:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T07:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T07:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T07:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-12T07:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T07:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T07:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T07:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T07:18:34.448 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-09-12T07:18:34.450 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-12T07:18:34.450 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-12T07:18:34.524 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-12T07:18:34.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:18:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:34.690+0000 7f958c41c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-12T07:18:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:34.690+0000 7f958c41c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-12T07:18:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:34.690+0000 7f958c41c640 -1 bdev(0x56294ce7d800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-12T07:18:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:34.690+0000 7f958c41c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-12T07:18:37.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-12T07:18:37.897 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:18:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:37.993+0000 7f7542145640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-12T07:18:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:37.993+0000 7f7542145640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-12T07:18:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:37.993+0000 7f7542145640 -1 bdev(0x55ec8ba21800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-12T07:18:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:37.994+0000 7f7542145640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-12T07:18:40.899 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-12T07:18:40.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:18:41.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:41.023+0000 7f3f11a10640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-12T07:18:41.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:41.023+0000 7f3f11a10640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-12T07:18:41.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:41.023+0000 7f3f11a10640 -1 bdev(0x5577f3a17800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-12T07:18:41.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:41.023+0000 7f3f11a10640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-12T07:18:43.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-12T07:18:43.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:18:44.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:44.055+0000 7fe7d8c2f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-12T07:18:44.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:44.055+0000 7fe7d8c2f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-12T07:18:44.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:44.055+0000 7fe7d8c2f640 -1 bdev(0x558b790bf800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-12T07:18:44.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:18:44.055+0000 7fe7d8c2f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-12T07:18:46.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-12T07:18:47.004 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-12T07:18:47.035 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T07:18:47.035 INFO:tasks.ceph:role: osd.4 2024-09-12T07:18:47.035 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T07:18:47.036 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-12T07:18:47.112 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T07:18:47.112 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T07:18:47.112 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T07:18:47.113 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T07:18:47.113 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T07:18:47.113 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-09-12T07:18:47.113 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T07:18:47.113 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T07:18:47.113 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T07:18:47.113 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T07:18:47.629 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2024-09-12T07:18:47.631 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-09-12T07:18:47.632 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-12T07:18:47.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-12T07:18:47.820 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-12T07:18:47.891 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T07:18:47.891 INFO:tasks.ceph:role: osd.5 2024-09-12T07:18:47.891 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T07:18:47.891 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-12T07:18:47.967 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T07:18:47.967 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T07:18:47.967 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T07:18:47.967 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T07:18:47.968 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T07:18:47.968 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-09-12T07:18:47.968 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T07:18:47.968 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T07:18:47.968 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T07:18:47.968 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T07:18:48.428 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2024-09-12T07:18:48.430 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-09-12T07:18:48.431 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-12T07:18:48.521 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-12T07:18:48.550 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-12T07:18:48.616 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T07:18:48.617 INFO:tasks.ceph:role: osd.6 2024-09-12T07:18:48.617 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T07:18:48.617 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-12T07:18:48.690 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T07:18:48.690 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T07:18:48.690 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T07:18:48.690 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T07:18:48.690 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T07:18:48.690 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-09-12T07:18:48.690 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T07:18:48.691 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T07:18:48.691 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T07:18:48.691 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T07:18:49.176 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2024-09-12T07:18:49.178 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-09-12T07:18:49.178 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-12T07:18:49.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-12T07:18:49.326 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-12T07:18:49.393 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T07:18:49.393 INFO:tasks.ceph:role: osd.7 2024-09-12T07:18:49.393 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T07:18:49.393 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-12T07:18:49.466 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T07:18:49.466 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T07:18:49.466 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T07:18:49.466 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T07:18:49.466 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T07:18:49.466 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-09-12T07:18:49.466 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T07:18:49.467 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T07:18:49.467 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T07:18:49.467 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T07:18:49.963 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2024-09-12T07:18:49.965 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-09-12T07:18:49.966 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-12T07:18:50.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-12T07:18:50.135 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:18:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:50.236+0000 7ff2a9d56640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-12T07:18:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:50.236+0000 7ff2a9d56640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-12T07:18:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:50.236+0000 7ff2a9d56640 -1 bdev(0x562499817800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-12T07:18:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:50.236+0000 7ff2a9d56640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-12T07:18:53.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-12T07:18:53.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:18:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:53.512+0000 7fb92fa24640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-12T07:18:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:53.512+0000 7fb92fa24640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-12T07:18:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:53.512+0000 7fb92fa24640 -1 bdev(0x5630916f7800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-12T07:18:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:53.512+0000 7fb92fa24640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-12T07:18:56.674 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-12T07:18:56.705 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:18:56.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:56.800+0000 7f0254a1c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-12T07:18:56.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:56.801+0000 7f0254a1c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-12T07:18:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:56.801+0000 7f0254a1c640 -1 bdev(0x55a660865800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-12T07:18:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:56.801+0000 7f0254a1c640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-12T07:18:59.716 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-12T07:18:59.747 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:18:59.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:59.840+0000 7fcba3c0e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-12T07:18:59.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:59.840+0000 7fcba3c0e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-12T07:18:59.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:59.840+0000 7fcba3c0e640 -1 bdev(0x55e157555800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-12T07:18:59.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T07:18:59.840+0000 7fcba3c0e640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-12T07:19:03.003 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-12T07:19:03.032 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-12T07:19:03.032 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:19:03.032 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-12T07:19:03.064 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:19:03.064 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-12T07:19:03.135 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:19:03.135 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-12T07:19:03.204 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:19:03.205 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-12T07:19:03.274 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:19:03.274 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-12T07:19:03.343 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:19:03.343 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-12T07:19:03.374 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:19:03.374 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-12T07:19:03.444 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:19:03.444 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-12T07:19:03.513 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:19:03.513 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-12T07:19:03.581 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:19:03.582 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-12T07:19:03.649 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:19:03.649 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-12T07:19:03.665 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:19:03.665 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-12T07:19:03.706 INFO:tasks.ceph:Adding keys to all mons... 2024-09-12T07:19:03.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-12T07:19:03.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-12T07:19:03.750 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.y] 2024-09-12T07:19:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDJleJmFM2TDRAAGQPu1TbhmphT+9rxbKi1mg== 2024-09-12T07:19:03.750 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.0] 2024-09-12T07:19:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDKleJmuJ4wKRAAJipOq4sDXHN9oWV4d8llxA== 2024-09-12T07:19:03.750 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.1] 2024-09-12T07:19:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDNleJmX5hHOxAAiHP+Lt7tjs80CO0USRkZxg== 2024-09-12T07:19:03.750 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.2] 2024-09-12T07:19:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDRleJm5S52ARAAYkBTl7vQQqMvbiQH4bhD/w== 2024-09-12T07:19:03.750 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.3] 2024-09-12T07:19:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDUleJmNHBdAxAAngRqMIKL6xbAU1685FsT6g== 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.x] 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDJleJmL/NyERAAHn70WfbQKxFCce7JHSvULg== 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.4] 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDaleJmtwMnDhAAlUdjJ157271luCDiV5P0xA== 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.5] 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDdleJmFgiTHhAAcCYd7e2fzMWSf4uZcKYvOw== 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.6] 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDgleJmMEPJLxAAU0kCvEmYB99fRrXAq+9e8g== 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.7] 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDjleJmk4QlMhAAluJWosQ2M//3jx4AvNu4Uw== 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDJleJmATs8FBAA10WxZcbYXYakHyuVAkTnhQ== 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout:[client.1] 2024-09-12T07:19:03.751 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDJleJmHovOFxAA1R9ntO8bUdTO8LNMrKDYHg== 2024-09-12T07:19:03.772 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.y] 2024-09-12T07:19:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDJleJmFM2TDRAAGQPu1TbhmphT+9rxbKi1mg== 2024-09-12T07:19:03.772 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.0] 2024-09-12T07:19:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDKleJmuJ4wKRAAJipOq4sDXHN9oWV4d8llxA== 2024-09-12T07:19:03.772 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.1] 2024-09-12T07:19:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDNleJmX5hHOxAAiHP+Lt7tjs80CO0USRkZxg== 2024-09-12T07:19:03.772 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.2] 2024-09-12T07:19:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDRleJm5S52ARAAYkBTl7vQQqMvbiQH4bhD/w== 2024-09-12T07:19:03.772 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.3] 2024-09-12T07:19:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDUleJmNHBdAxAAngRqMIKL6xbAU1685FsT6g== 2024-09-12T07:19:03.772 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.x] 2024-09-12T07:19:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDJleJmL/NyERAAHn70WfbQKxFCce7JHSvULg== 2024-09-12T07:19:03.773 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.4] 2024-09-12T07:19:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDaleJmtwMnDhAAlUdjJ157271luCDiV5P0xA== 2024-09-12T07:19:03.773 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.5] 2024-09-12T07:19:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDdleJmFgiTHhAAcCYd7e2fzMWSf4uZcKYvOw== 2024-09-12T07:19:03.773 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.6] 2024-09-12T07:19:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDgleJmMEPJLxAAU0kCvEmYB99fRrXAq+9e8g== 2024-09-12T07:19:03.773 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.7] 2024-09-12T07:19:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDjleJmk4QlMhAAluJWosQ2M//3jx4AvNu4Uw== 2024-09-12T07:19:03.773 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2024-09-12T07:19:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDJleJmATs8FBAA10WxZcbYXYakHyuVAkTnhQ== 2024-09-12T07:19:03.773 INFO:teuthology.orchestra.run.smithi064.stdout:[client.1] 2024-09-12T07:19:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDJleJmHovOFxAA1R9ntO8bUdTO8LNMrKDYHg== 2024-09-12T07:19:03.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T07:19:03.795 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T07:19:03.869 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:03.872 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:03.925 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:03.928 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:03.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:03.983 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:04.035 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:04.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:04.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T07:19:04.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T07:19:04.144 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:04.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:04.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:04.200 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:04.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:04.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:04.306 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:04.309 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T07:19:04.361 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T07:19:04.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T07:19:04.418 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T07:19:04.421 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T07:19:04.476 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-12T07:19:04.476 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-12T07:19:04.501 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T07:19:04.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-12T07:19:04.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-12T07:19:04.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T07:19:04.786 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-12T07:19:04.812 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-12T07:19:04.838 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T07:19:04.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-12T07:19:04.963 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:19:04.965 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-12T07:19:05.019 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-12T07:19:05.019 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-12T07:19:05.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-12T07:19:05.022 INFO:tasks.ceph.mon.a:Started 2024-09-12T07:19:05.022 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-12T07:19:05.022 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-12T07:19:05.024 INFO:tasks.ceph.mon.c:Started 2024-09-12T07:19:05.024 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-12T07:19:05.024 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-12T07:19:05.061 INFO:tasks.ceph.mon.b:Started 2024-09-12T07:19:05.061 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-12T07:19:05.062 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-12T07:19:05.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-12T07:19:05.065 INFO:tasks.ceph.mgr.y:Started 2024-09-12T07:19:05.065 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-12T07:19:05.065 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-12T07:19:05.067 INFO:tasks.ceph.mgr.x:Started 2024-09-12T07:19:05.067 DEBUG:tasks.ceph:set 0 configs 2024-09-12T07:19:05.067 INFO:tasks.ceph:Setting crush tunables to default 2024-09-12T07:19:05.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd crush tunables default 2024-09-12T07:19:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-09-12T07:19:10.436 INFO:tasks.ceph:check_enable_crimson: False 2024-09-12T07:19:10.436 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-12T07:19:10.437 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:19:10.437 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-12T07:19:10.463 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:19:10.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-12T07:19:10.530 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:19:10.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-12T07:19:10.597 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T07:19:10.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-12T07:19:10.663 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:19:10.663 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-12T07:19:10.696 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:19:10.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-12T07:19:10.766 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:19:10.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-12T07:19:10.835 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T07:19:10.835 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-12T07:19:10.904 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new 710c25c3-92d3-4b51-a7fc-670879faa69c 0 2024-09-12T07:19:11.132 INFO:teuthology.orchestra.run.smithi064.stdout:0 2024-09-12T07:19:11.144 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new 04479191-c52e-4b60-aa0d-3e911f869062 1 2024-09-12T07:19:11.355 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-09-12T07:19:11.367 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new ba673e51-96aa-424b-a4df-26432d633ebe 2 2024-09-12T07:19:11.550 INFO:teuthology.orchestra.run.smithi064.stdout:2 2024-09-12T07:19:11.561 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new c516e59c-834c-4fb0-9ff8-bcc80b21d28a 3 2024-09-12T07:19:11.749 INFO:teuthology.orchestra.run.smithi064.stdout:3 2024-09-12T07:19:11.760 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new 450a70cd-ebb2-48ed-a766-184f58e7032e 4 2024-09-12T07:19:11.945 INFO:teuthology.orchestra.run.smithi064.stdout:4 2024-09-12T07:19:11.955 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new a4efa333-bf57-45e7-a281-1d3bc9d2e82b 5 2024-09-12T07:19:12.140 INFO:teuthology.orchestra.run.smithi064.stdout:5 2024-09-12T07:19:12.151 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new bcaad352-f9ee-426c-bcbf-39c8d031aae0 6 2024-09-12T07:19:12.337 INFO:teuthology.orchestra.run.smithi064.stdout:6 2024-09-12T07:19:12.353 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new 97e5ef36-58a4-4667-9f9e-b8ced5904d69 7 2024-09-12T07:19:12.537 INFO:teuthology.orchestra.run.smithi064.stdout:7 2024-09-12T07:19:12.548 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-12T07:19:12.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-12T07:19:12.552 INFO:tasks.ceph.osd.0:Started 2024-09-12T07:19:12.552 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-12T07:19:12.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-12T07:19:12.554 INFO:tasks.ceph.osd.1:Started 2024-09-12T07:19:12.554 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-12T07:19:12.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-12T07:19:12.556 INFO:tasks.ceph.osd.2:Started 2024-09-12T07:19:12.557 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-12T07:19:12.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-12T07:19:12.559 INFO:tasks.ceph.osd.3:Started 2024-09-12T07:19:12.559 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-12T07:19:12.559 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-12T07:19:12.561 INFO:tasks.ceph.osd.4:Started 2024-09-12T07:19:12.561 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-12T07:19:12.561 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-12T07:19:12.563 INFO:tasks.ceph.osd.5:Started 2024-09-12T07:19:12.563 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-12T07:19:12.563 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-12T07:19:12.565 INFO:tasks.ceph.osd.6:Started 2024-09-12T07:19:12.565 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-12T07:19:12.565 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-12T07:19:12.567 INFO:tasks.ceph.osd.7:Started 2024-09-12T07:19:12.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T07:19:12.764 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:19:12.764 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"fsid":"8412ed58-ac5e-4549-b523-1a58cf9f9429","created":"2024-09-12T07:19:10.328057+0000","modified":"2024-09-12T07:19:12.537858+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T07:19:12.537858+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"710c25c3-92d3-4b51-a7fc-670879faa69c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"04479191-c52e-4b60-aa0d-3e911f869062","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ba673e51-96aa-424b-a4df-26432d633ebe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c516e59c-834c-4fb0-9ff8-bcc80b21d28a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"450a70cd-ebb2-48ed-a766-184f58e7032e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a4efa333-bf57-45e7-a281-1d3bc9d2e82b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"bcaad352-f9ee-426c-bcbf-39c8d031aae0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"97e5ef36-58a4-4667-9f9e-b8ced5904d69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T07:19:12.779 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-12T07:19:12.779 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-12T07:19:13.536 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:13.537+0000 7fd5bc411640 -1 Falling back to public interface 2024-09-12T07:19:13.542 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:13.543+0000 7f106860e640 -1 Falling back to public interface 2024-09-12T07:19:13.585 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T07:19:13.751 INFO:teuthology.misc.health.smithi046.stdout: 2024-09-12T07:19:13.751 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":10,"fsid":"8412ed58-ac5e-4549-b523-1a58cf9f9429","created":"2024-09-12T07:19:10.328057+0000","modified":"2024-09-12T07:19:12.537858+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T07:19:12.537858+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"710c25c3-92d3-4b51-a7fc-670879faa69c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"04479191-c52e-4b60-aa0d-3e911f869062","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ba673e51-96aa-424b-a4df-26432d633ebe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c516e59c-834c-4fb0-9ff8-bcc80b21d28a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"450a70cd-ebb2-48ed-a766-184f58e7032e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a4efa333-bf57-45e7-a281-1d3bc9d2e82b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"bcaad352-f9ee-426c-bcbf-39c8d031aae0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"97e5ef36-58a4-4667-9f9e-b8ced5904d69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T07:19:13.761 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-12T07:19:14.023 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:14.024+0000 7f63bf615640 -1 Falling back to public interface 2024-09-12T07:19:14.281 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:14.282+0000 7f292ae16640 -1 Falling back to public interface 2024-09-12T07:19:14.283 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:14.284+0000 7f0cb8233640 -1 Falling back to public interface 2024-09-12T07:19:14.283 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:14.284+0000 7f904d41c640 -1 Falling back to public interface 2024-09-12T07:19:14.283 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:14.285+0000 7f0cb71db640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6803/0: (98) Address already in use 2024-09-12T07:19:14.284 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:14.286+0000 7f0cb71db640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6812/0: (98) Address already in use 2024-09-12T07:19:14.511 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:14.512+0000 7f0de1e24640 -1 Falling back to public interface 2024-09-12T07:19:14.518 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:14.519+0000 7f595fa0d640 -1 Falling back to public interface 2024-09-12T07:19:16.495 INFO:tasks.ceph.mgr.x.smithi064.stderr:2024-09-12T07:19:16.496+0000 7f8716b9e640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-12T07:19:16.495 INFO:tasks.ceph.mgr.x.smithi064.stderr:2024-09-12T07:19:16.496+0000 7f8716b9e640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-12T07:19:17.614 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:17.615+0000 7fd5bc411640 -1 osd.2 0 log_to_monitors true 2024-09-12T07:19:17.833 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:17.835+0000 7f63bf615640 -1 osd.0 0 log_to_monitors true 2024-09-12T07:19:17.881 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:17.882+0000 7f106860e640 -1 osd.3 0 log_to_monitors true 2024-09-12T07:19:18.063 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:18.064+0000 7f292ae16640 -1 osd.1 0 log_to_monitors true 2024-09-12T07:19:18.071 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:18.073+0000 7f595fa0d640 -1 osd.5 0 log_to_monitors true 2024-09-12T07:19:18.099 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:18.100+0000 7f904d41c640 -1 osd.7 0 log_to_monitors true 2024-09-12T07:19:18.509 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:18.511+0000 7f0de1e24640 -1 osd.4 0 log_to_monitors true 2024-09-12T07:19:18.573 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:18.575+0000 7f0cb8233640 -1 osd.6 0 log_to_monitors true 2024-09-12T07:19:20.568 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T07:19:21.148 INFO:teuthology.misc.health.smithi046.stdout: 2024-09-12T07:19:21.149 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":13,"fsid":"8412ed58-ac5e-4549-b523-1a58cf9f9429","created":"2024-09-12T07:19:10.328057+0000","modified":"2024-09-12T07:19:20.547703+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T07:19:12.537858+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"710c25c3-92d3-4b51-a7fc-670879faa69c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"04479191-c52e-4b60-aa0d-3e911f869062","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ba673e51-96aa-424b-a4df-26432d633ebe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c516e59c-834c-4fb0-9ff8-bcc80b21d28a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"450a70cd-ebb2-48ed-a766-184f58e7032e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a4efa333-bf57-45e7-a281-1d3bc9d2e82b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"bcaad352-f9ee-426c-bcbf-39c8d031aae0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"97e5ef36-58a4-4667-9f9e-b8ced5904d69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T07:19:21.167 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-12T07:19:21.173 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:21.172+0000 7f1063efe640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T07:19:21.185 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:21.186+0000 7f105bd1d640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T07:19:21.208 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:21.208+0000 7f292679a640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T07:19:21.220 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:21.221+0000 7f291e5b9640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T07:19:21.261 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:21.262+0000 7fd5b7d1e640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T07:19:21.277 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:21.278+0000 7fd5afb3d640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T07:19:21.294 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:21.294+0000 7f63baf26640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T07:19:21.304 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:21.305+0000 7f63b2d45640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T07:19:21.562 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:21.563+0000 7f0ddd80f640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T07:19:21.579 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:21.580+0000 7f0dd562e640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T07:19:21.641 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:21.642+0000 7f9048e5a640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T07:19:21.645 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:21.646+0000 7f595b44b640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T07:19:21.655 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:21.657+0000 7f9040c79640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T07:19:21.658 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:21.660+0000 7f595326a640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T07:19:21.834 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:21.835+0000 7f0cb41c2640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T07:19:21.852 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:21.854+0000 7f0cabfe1640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T07:19:27.973 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T07:19:28.278 INFO:teuthology.misc.health.smithi046.stdout: 2024-09-12T07:19:28.278 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":18,"fsid":"8412ed58-ac5e-4549-b523-1a58cf9f9429","created":"2024-09-12T07:19:10.328057+0000","modified":"2024-09-12T07:19:26.186621+0000","last_up_change":"2024-09-12T07:19:22.177473+0000","last_in_change":"2024-09-12T07:19:12.537858+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T07:19:23.507151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"710c25c3-92d3-4b51-a7fc-670879faa69c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6817","nonce":4081287959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6819","nonce":4081287959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6823","nonce":4081287959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6821","nonce":4081287959}]},"public_addr":"172.21.15.46:6817/4081287959","cluster_addr":"172.21.15.46:6819/4081287959","heartbeat_back_addr":"172.21.15.46:6823/4081287959","heartbeat_front_addr":"172.21.15.46:6821/4081287959","state":["exists","up"]},{"osd":1,"uuid":"04479191-c52e-4b60-aa0d-3e911f869062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6825","nonce":3980488213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6827","nonce":3980488213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6831","nonce":3980488213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6829","nonce":3980488213}]},"public_addr":"172.21.15.46:6825/3980488213","cluster_addr":"172.21.15.46:6827/3980488213","heartbeat_back_addr":"172.21.15.46:6831/3980488213","heartbeat_front_addr":"172.21.15.46:6829/3980488213","state":["exists","up"]},{"osd":2,"uuid":"ba673e51-96aa-424b-a4df-26432d633ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":656802655},{"type":"v1","addr":"172.21.15.46:6801","nonce":656802655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":656802655},{"type":"v1","addr":"172.21.15.46:6803","nonce":656802655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":656802655},{"type":"v1","addr":"172.21.15.46:6807","nonce":656802655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":656802655},{"type":"v1","addr":"172.21.15.46:6805","nonce":656802655}]},"public_addr":"172.21.15.46:6801/656802655","cluster_addr":"172.21.15.46:6803/656802655","heartbeat_back_addr":"172.21.15.46:6807/656802655","heartbeat_front_addr":"172.21.15.46:6805/656802655","state":["exists","up"]},{"osd":3,"uuid":"c516e59c-834c-4fb0-9ff8-bcc80b21d28a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6809","nonce":3848260327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6811","nonce":3848260327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6815","nonce":3848260327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6813","nonce":3848260327}]},"public_addr":"172.21.15.46:6809/3848260327","cluster_addr":"172.21.15.46:6811/3848260327","heartbeat_back_addr":"172.21.15.46:6815/3848260327","heartbeat_front_addr":"172.21.15.46:6813/3848260327","state":["exists","up"]},{"osd":4,"uuid":"450a70cd-ebb2-48ed-a766-184f58e7032e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6817","nonce":3464256744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6819","nonce":3464256744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6823","nonce":3464256744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6821","nonce":3464256744}]},"public_addr":"172.21.15.64:6817/3464256744","cluster_addr":"172.21.15.64:6819/3464256744","heartbeat_back_addr":"172.21.15.64:6823/3464256744","heartbeat_front_addr":"172.21.15.64:6821/3464256744","state":["exists","up"]},{"osd":5,"uuid":"a4efa333-bf57-45e7-a281-1d3bc9d2e82b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6825","nonce":1789414693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6827","nonce":1789414693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6831","nonce":1789414693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6829","nonce":1789414693}]},"public_addr":"172.21.15.64:6825/1789414693","cluster_addr":"172.21.15.64:6827/1789414693","heartbeat_back_addr":"172.21.15.64:6831/1789414693","heartbeat_front_addr":"172.21.15.64:6829/1789414693","state":["exists","up"]},{"osd":6,"uuid":"bcaad352-f9ee-426c-bcbf-39c8d031aae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6801","nonce":2576774646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6804","nonce":2576774646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6813","nonce":2576774646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6808","nonce":2576774646}]},"public_addr":"172.21.15.64:6801/2576774646","cluster_addr":"172.21.15.64:6804/2576774646","heartbeat_back_addr":"172.21.15.64:6813/2576774646","heartbeat_front_addr":"172.21.15.64:6808/2576774646","state":["exists","up"]},{"osd":7,"uuid":"97e5ef36-58a4-4667-9f9e-b8ced5904d69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6803","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6805","nonce":368918691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6807","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6809","nonce":368918691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6815","nonce":368918691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6811","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6812","nonce":368918691}]},"public_addr":"172.21.15.64:6805/368918691","cluster_addr":"172.21.15.64:6809/368918691","heartbeat_back_addr":"172.21.15.64:6815/368918691","heartbeat_front_addr":"172.21.15.64:6812/368918691","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.862785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.059718+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.575165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.847666+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.471904+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.061712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.541383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.072272+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T07:19:28.289 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-12T07:19:28.289 INFO:tasks.ceph:Creating RBD pool 2024-09-12T07:19:28.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-12T07:19:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'rbd' created 2024-09-12T07:19:29.219 DEBUG:teuthology.orchestra.run.smithi046:> rbd --cluster ceph pool init rbd 2024-09-12T07:19:32.233 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-12T07:19:32.234 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-12T07:19:32.234 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-12T07:19:32.544 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:19:32.557 INFO:teuthology.orchestra.run.smithi046.stdout:[{"version":9,"timestamp":"2024-09-12T07:19:21.837327+0000","name":"","changes":[{"name":"osd.6/osd_mclock_max_capacity_iops_ssd","new_value":"4617.797009"}]}] 2024-09-12T07:19:32.557 INFO:tasks.ceph_manager:config epoch is 9 2024-09-12T07:19:32.557 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T07:19:32.557 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-12T07:19:32.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-12T07:19:32.908 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:19:32.918 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"flags":0,"active_gid":4120,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3868747909},{"type":"v1","addr":"172.21.15.64:6833","nonce":3868747909}]},"active_addr":"172.21.15.64:6833/3868747909","active_change":"2024-09-12T07:19:15.481761+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4112,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":4024520985}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":4186274146}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2623724654}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":1634836670}]}]} 2024-09-12T07:19:32.923 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-12T07:19:32.923 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-12T07:19:32.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T07:19:33.223 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:19:33.224 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"8412ed58-ac5e-4549-b523-1a58cf9f9429","created":"2024-09-12T07:19:10.328057+0000","modified":"2024-09-12T07:19:32.215704+0000","last_up_change":"2024-09-12T07:19:22.177473+0000","last_in_change":"2024-09-12T07:19:12.537858+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T07:19:23.507151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T07:19:28.593632+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"710c25c3-92d3-4b51-a7fc-670879faa69c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6817","nonce":4081287959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6819","nonce":4081287959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6823","nonce":4081287959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6821","nonce":4081287959}]},"public_addr":"172.21.15.46:6817/4081287959","cluster_addr":"172.21.15.46:6819/4081287959","heartbeat_back_addr":"172.21.15.46:6823/4081287959","heartbeat_front_addr":"172.21.15.46:6821/4081287959","state":["exists","up"]},{"osd":1,"uuid":"04479191-c52e-4b60-aa0d-3e911f869062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6825","nonce":3980488213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6827","nonce":3980488213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6831","nonce":3980488213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6829","nonce":3980488213}]},"public_addr":"172.21.15.46:6825/3980488213","cluster_addr":"172.21.15.46:6827/3980488213","heartbeat_back_addr":"172.21.15.46:6831/3980488213","heartbeat_front_addr":"172.21.15.46:6829/3980488213","state":["exists","up"]},{"osd":2,"uuid":"ba673e51-96aa-424b-a4df-26432d633ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":656802655},{"type":"v1","addr":"172.21.15.46:6801","nonce":656802655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":656802655},{"type":"v1","addr":"172.21.15.46:6803","nonce":656802655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":656802655},{"type":"v1","addr":"172.21.15.46:6807","nonce":656802655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":656802655},{"type":"v1","addr":"172.21.15.46:6805","nonce":656802655}]},"public_addr":"172.21.15.46:6801/656802655","cluster_addr":"172.21.15.46:6803/656802655","heartbeat_back_addr":"172.21.15.46:6807/656802655","heartbeat_front_addr":"172.21.15.46:6805/656802655","state":["exists","up"]},{"osd":3,"uuid":"c516e59c-834c-4fb0-9ff8-bcc80b21d28a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6809","nonce":3848260327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6811","nonce":3848260327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6815","nonce":3848260327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6813","nonce":3848260327}]},"public_addr":"172.21.15.46:6809/3848260327","cluster_addr":"172.21.15.46:6811/3848260327","heartbeat_back_addr":"172.21.15.46:6815/3848260327","heartbeat_front_addr":"172.21.15.46:6813/3848260327","state":["exists","up"]},{"osd":4,"uuid":"450a70cd-ebb2-48ed-a766-184f58e7032e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6817","nonce":3464256744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6819","nonce":3464256744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6823","nonce":3464256744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6821","nonce":3464256744}]},"public_addr":"172.21.15.64:6817/3464256744","cluster_addr":"172.21.15.64:6819/3464256744","heartbeat_back_addr":"172.21.15.64:6823/3464256744","heartbeat_front_addr":"172.21.15.64:6821/3464256744","state":["exists","up"]},{"osd":5,"uuid":"a4efa333-bf57-45e7-a281-1d3bc9d2e82b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6825","nonce":1789414693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6827","nonce":1789414693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6831","nonce":1789414693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6829","nonce":1789414693}]},"public_addr":"172.21.15.64:6825/1789414693","cluster_addr":"172.21.15.64:6827/1789414693","heartbeat_back_addr":"172.21.15.64:6831/1789414693","heartbeat_front_addr":"172.21.15.64:6829/1789414693","state":["exists","up"]},{"osd":6,"uuid":"bcaad352-f9ee-426c-bcbf-39c8d031aae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6801","nonce":2576774646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6804","nonce":2576774646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6813","nonce":2576774646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6808","nonce":2576774646}]},"public_addr":"172.21.15.64:6801/2576774646","cluster_addr":"172.21.15.64:6804/2576774646","heartbeat_back_addr":"172.21.15.64:6813/2576774646","heartbeat_front_addr":"172.21.15.64:6808/2576774646","state":["exists","up"]},{"osd":7,"uuid":"97e5ef36-58a4-4667-9f9e-b8ced5904d69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6803","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6805","nonce":368918691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6807","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6809","nonce":368918691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6815","nonce":368918691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6811","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6812","nonce":368918691}]},"public_addr":"172.21.15.64:6805/368918691","cluster_addr":"172.21.15.64:6809/368918691","heartbeat_back_addr":"172.21.15.64:6815/368918691","heartbeat_front_addr":"172.21.15.64:6812/368918691","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.862785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.059718+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.575165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.847666+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.471904+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.061712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.541383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.072272+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T07:19:33.237 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-12T07:19:33.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T07:19:33.535 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:19:33.535 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"8412ed58-ac5e-4549-b523-1a58cf9f9429","created":"2024-09-12T07:19:10.328057+0000","modified":"2024-09-12T07:19:32.215704+0000","last_up_change":"2024-09-12T07:19:22.177473+0000","last_in_change":"2024-09-12T07:19:12.537858+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T07:19:23.507151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T07:19:28.593632+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"710c25c3-92d3-4b51-a7fc-670879faa69c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6817","nonce":4081287959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6819","nonce":4081287959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6823","nonce":4081287959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6821","nonce":4081287959}]},"public_addr":"172.21.15.46:6817/4081287959","cluster_addr":"172.21.15.46:6819/4081287959","heartbeat_back_addr":"172.21.15.46:6823/4081287959","heartbeat_front_addr":"172.21.15.46:6821/4081287959","state":["exists","up"]},{"osd":1,"uuid":"04479191-c52e-4b60-aa0d-3e911f869062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6825","nonce":3980488213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6827","nonce":3980488213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6831","nonce":3980488213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6829","nonce":3980488213}]},"public_addr":"172.21.15.46:6825/3980488213","cluster_addr":"172.21.15.46:6827/3980488213","heartbeat_back_addr":"172.21.15.46:6831/3980488213","heartbeat_front_addr":"172.21.15.46:6829/3980488213","state":["exists","up"]},{"osd":2,"uuid":"ba673e51-96aa-424b-a4df-26432d633ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":656802655},{"type":"v1","addr":"172.21.15.46:6801","nonce":656802655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":656802655},{"type":"v1","addr":"172.21.15.46:6803","nonce":656802655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":656802655},{"type":"v1","addr":"172.21.15.46:6807","nonce":656802655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":656802655},{"type":"v1","addr":"172.21.15.46:6805","nonce":656802655}]},"public_addr":"172.21.15.46:6801/656802655","cluster_addr":"172.21.15.46:6803/656802655","heartbeat_back_addr":"172.21.15.46:6807/656802655","heartbeat_front_addr":"172.21.15.46:6805/656802655","state":["exists","up"]},{"osd":3,"uuid":"c516e59c-834c-4fb0-9ff8-bcc80b21d28a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6809","nonce":3848260327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6811","nonce":3848260327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6815","nonce":3848260327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6813","nonce":3848260327}]},"public_addr":"172.21.15.46:6809/3848260327","cluster_addr":"172.21.15.46:6811/3848260327","heartbeat_back_addr":"172.21.15.46:6815/3848260327","heartbeat_front_addr":"172.21.15.46:6813/3848260327","state":["exists","up"]},{"osd":4,"uuid":"450a70cd-ebb2-48ed-a766-184f58e7032e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6817","nonce":3464256744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6819","nonce":3464256744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6823","nonce":3464256744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6821","nonce":3464256744}]},"public_addr":"172.21.15.64:6817/3464256744","cluster_addr":"172.21.15.64:6819/3464256744","heartbeat_back_addr":"172.21.15.64:6823/3464256744","heartbeat_front_addr":"172.21.15.64:6821/3464256744","state":["exists","up"]},{"osd":5,"uuid":"a4efa333-bf57-45e7-a281-1d3bc9d2e82b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6825","nonce":1789414693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6827","nonce":1789414693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6831","nonce":1789414693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6829","nonce":1789414693}]},"public_addr":"172.21.15.64:6825/1789414693","cluster_addr":"172.21.15.64:6827/1789414693","heartbeat_back_addr":"172.21.15.64:6831/1789414693","heartbeat_front_addr":"172.21.15.64:6829/1789414693","state":["exists","up"]},{"osd":6,"uuid":"bcaad352-f9ee-426c-bcbf-39c8d031aae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6801","nonce":2576774646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6804","nonce":2576774646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6813","nonce":2576774646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6808","nonce":2576774646}]},"public_addr":"172.21.15.64:6801/2576774646","cluster_addr":"172.21.15.64:6804/2576774646","heartbeat_back_addr":"172.21.15.64:6813/2576774646","heartbeat_front_addr":"172.21.15.64:6808/2576774646","state":["exists","up"]},{"osd":7,"uuid":"97e5ef36-58a4-4667-9f9e-b8ced5904d69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6803","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6805","nonce":368918691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6807","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6809","nonce":368918691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6815","nonce":368918691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6811","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6812","nonce":368918691}]},"public_addr":"172.21.15.64:6805/368918691","cluster_addr":"172.21.15.64:6809/368918691","heartbeat_back_addr":"172.21.15.64:6815/368918691","heartbeat_front_addr":"172.21.15.64:6812/368918691","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.862785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.059718+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.575165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.847666+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.471904+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.061712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.541383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.072272+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T07:19:33.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-12T07:19:33.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-12T07:19:33.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-12T07:19:33.550 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-12T07:19:33.550 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-12T07:19:33.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-12T07:19:33.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-12T07:19:33.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-12T07:19:33.794 INFO:teuthology.orchestra.run.smithi046.stdout:60129542150 2024-09-12T07:19:33.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T07:19:33.804 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:33.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T07:19:33.805 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:33.805 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:33.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T07:19:33.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T07:19:33.807 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:33.807 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T07:19:33.808 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:33.808 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:33.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T07:19:33.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T07:19:33.810 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:33.811 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T07:19:34.371 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:34.391 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2024-09-12T07:19:34.425 INFO:teuthology.orchestra.run.smithi046.stdout:60129542147 2024-09-12T07:19:34.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2024-09-12T07:19:34.472 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:34.482 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:34.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.0 2024-09-12T07:19:34.490 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:34.490 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:34.494 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:34.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.3 2024-09-12T07:19:34.499 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:34.508 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.7 2024-09-12T07:19:34.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.5 2024-09-12T07:19:34.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.2 2024-09-12T07:19:34.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2024-09-12T07:19:35.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T07:19:35.440 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T07:19:35.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T07:19:35.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T07:19:35.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T07:19:35.510 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T07:19:35.511 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T07:19:35.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T07:19:35.917 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:35.937 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2024-09-12T07:19:36.018 INFO:teuthology.orchestra.run.smithi046.stdout:60129542147 2024-09-12T07:19:36.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2024-09-12T07:19:36.040 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:36.054 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.7 2024-09-12T07:19:36.054 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:36.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.0 2024-09-12T07:19:36.133 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:36.136 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:36.137 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:36.141 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:36.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.2 2024-09-12T07:19:36.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2024-09-12T07:19:36.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.3 2024-09-12T07:19:36.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.5 2024-09-12T07:19:36.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T07:19:37.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T07:19:37.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T07:19:37.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T07:19:37.150 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T07:19:37.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T07:19:37.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T07:19:37.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T07:19:37.365 INFO:teuthology.orchestra.run.smithi046.stdout:60129542150 2024-09-12T07:19:37.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2024-09-12T07:19:37.382 DEBUG:teuthology.parallel:result is None 2024-09-12T07:19:37.520 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:37.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.0 2024-09-12T07:19:37.534 DEBUG:teuthology.parallel:result is None 2024-09-12T07:19:37.566 INFO:teuthology.orchestra.run.smithi046.stdout:60129542148 2024-09-12T07:19:37.584 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.4 2024-09-12T07:19:37.585 DEBUG:teuthology.parallel:result is None 2024-09-12T07:19:37.610 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:37.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2024-09-12T07:19:37.625 DEBUG:teuthology.parallel:result is None 2024-09-12T07:19:37.661 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:37.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2024-09-12T07:19:37.675 DEBUG:teuthology.parallel:result is None 2024-09-12T07:19:37.695 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:37.703 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:37.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2024-09-12T07:19:37.708 DEBUG:teuthology.parallel:result is None 2024-09-12T07:19:37.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2024-09-12T07:19:37.717 DEBUG:teuthology.parallel:result is None 2024-09-12T07:19:37.732 INFO:teuthology.orchestra.run.smithi046.stdout:60129542149 2024-09-12T07:19:37.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.5 2024-09-12T07:19:37.746 DEBUG:teuthology.parallel:result is None 2024-09-12T07:19:37.746 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T07:19:37.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:19:38.081 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:19:38.081 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:19:38.093 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-12T07:19:37.492504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235796,"kb_used_data":2904,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738924,"statfs":{"total":773094113280,"available":772852658176,"internally_reserved":0,"allocated":2973696,"data_stored":1784070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.281985"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.474512+0000","last_change":"2024-09-12T07:19:32.474512+0000","last_active":"2024-09-12T07:19:32.474512+0000","last_peered":"2024-09-12T07:19:32.474512+0000","last_clean":"2024-09-12T07:19:32.474512+0000","last_became_active":"2024-09-12T07:19:30.219990+0000","last_became_peered":"2024-09-12T07:19:30.219990+0000","last_unstale":"2024-09-12T07:19:32.474512+0000","last_undegraded":"2024-09-12T07:19:32.474512+0000","last_fullsized":"2024-09-12T07:19:32.474512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:32.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046339800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.228066+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:19:32.228066+0000","last_peered":"2024-09-12T07:19:32.228066+0000","last_clean":"2024-09-12T07:19:32.228066+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:19:32.228066+0000","last_undegraded":"2024-09-12T07:19:32.228066+0000","last_fullsized":"2024-09-12T07:19:32.228066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:33.033060+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:19:33.033060+0000","last_peered":"2024-09-12T07:19:33.033060+0000","last_clean":"2024-09-12T07:19:33.033060+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:19:33.033060+0000","last_undegraded":"2024-09-12T07:19:33.033060+0000","last_fullsized":"2024-09-12T07:19:33.033060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.228114+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:19:32.228114+0000","last_peered":"2024-09-12T07:19:32.228114+0000","last_clean":"2024-09-12T07:19:32.228114+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:19:32.228114+0000","last_undegraded":"2024-09-12T07:19:32.228114+0000","last_fullsized":"2024-09-12T07:19:32.228114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.227049+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:19:32.227049+0000","last_peered":"2024-09-12T07:19:32.227049+0000","last_clean":"2024-09-12T07:19:32.227049+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:19:32.227049+0000","last_undegraded":"2024-09-12T07:19:32.227049+0000","last_fullsized":"2024-09-12T07:19:32.227049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.754632+0000","last_change":"2024-09-12T07:19:32.754826+0000","last_active":"2024-09-12T07:19:32.754632+0000","last_peered":"2024-09-12T07:19:32.754632+0000","last_clean":"2024-09-12T07:19:32.754632+0000","last_became_active":"2024-09-12T07:19:30.217328+0000","last_became_peered":"2024-09-12T07:19:30.217328+0000","last_unstale":"2024-09-12T07:19:32.754632+0000","last_undegraded":"2024-09-12T07:19:32.754632+0000","last_fullsized":"2024-09-12T07:19:32.754632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:57.208282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:33.033122+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:19:33.033122+0000","last_peered":"2024-09-12T07:19:33.033122+0000","last_clean":"2024-09-12T07:19:33.033122+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:19:33.033122+0000","last_undegraded":"2024-09-12T07:19:33.033122+0000","last_fullsized":"2024-09-12T07:19:33.033122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.225708+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:19:32.225708+0000","last_peered":"2024-09-12T07:19:32.225708+0000","last_clean":"2024-09-12T07:19:32.225708+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:19:32.225708+0000","last_undegraded":"2024-09-12T07:19:32.225708+0000","last_fullsized":"2024-09-12T07:19:32.225708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:33.032725+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:19:33.032725+0000","last_peered":"2024-09-12T07:19:33.032725+0000","last_clean":"2024-09-12T07:19:33.032725+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:19:33.032725+0000","last_undegraded":"2024-09-12T07:19:33.032725+0000","last_fullsized":"2024-09-12T07:19:33.032725+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":806912,"data_stored":661406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":667238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:19:38.094 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:19:38.394 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:19:38.394 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:19:38.406 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-12T07:19:37.492504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235796,"kb_used_data":2904,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738924,"statfs":{"total":773094113280,"available":772852658176,"internally_reserved":0,"allocated":2973696,"data_stored":1784070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.281985"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.474512+0000","last_change":"2024-09-12T07:19:32.474512+0000","last_active":"2024-09-12T07:19:32.474512+0000","last_peered":"2024-09-12T07:19:32.474512+0000","last_clean":"2024-09-12T07:19:32.474512+0000","last_became_active":"2024-09-12T07:19:30.219990+0000","last_became_peered":"2024-09-12T07:19:30.219990+0000","last_unstale":"2024-09-12T07:19:32.474512+0000","last_undegraded":"2024-09-12T07:19:32.474512+0000","last_fullsized":"2024-09-12T07:19:32.474512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:32.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046339800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.228066+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:19:32.228066+0000","last_peered":"2024-09-12T07:19:32.228066+0000","last_clean":"2024-09-12T07:19:32.228066+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:19:32.228066+0000","last_undegraded":"2024-09-12T07:19:32.228066+0000","last_fullsized":"2024-09-12T07:19:32.228066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:33.033060+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:19:33.033060+0000","last_peered":"2024-09-12T07:19:33.033060+0000","last_clean":"2024-09-12T07:19:33.033060+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:19:33.033060+0000","last_undegraded":"2024-09-12T07:19:33.033060+0000","last_fullsized":"2024-09-12T07:19:33.033060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.228114+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:19:32.228114+0000","last_peered":"2024-09-12T07:19:32.228114+0000","last_clean":"2024-09-12T07:19:32.228114+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:19:32.228114+0000","last_undegraded":"2024-09-12T07:19:32.228114+0000","last_fullsized":"2024-09-12T07:19:32.228114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.227049+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:19:32.227049+0000","last_peered":"2024-09-12T07:19:32.227049+0000","last_clean":"2024-09-12T07:19:32.227049+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:19:32.227049+0000","last_undegraded":"2024-09-12T07:19:32.227049+0000","last_fullsized":"2024-09-12T07:19:32.227049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.754632+0000","last_change":"2024-09-12T07:19:32.754826+0000","last_active":"2024-09-12T07:19:32.754632+0000","last_peered":"2024-09-12T07:19:32.754632+0000","last_clean":"2024-09-12T07:19:32.754632+0000","last_became_active":"2024-09-12T07:19:30.217328+0000","last_became_peered":"2024-09-12T07:19:30.217328+0000","last_unstale":"2024-09-12T07:19:32.754632+0000","last_undegraded":"2024-09-12T07:19:32.754632+0000","last_fullsized":"2024-09-12T07:19:32.754632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:57.208282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:33.033122+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:19:33.033122+0000","last_peered":"2024-09-12T07:19:33.033122+0000","last_clean":"2024-09-12T07:19:33.033122+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:19:33.033122+0000","last_undegraded":"2024-09-12T07:19:33.033122+0000","last_fullsized":"2024-09-12T07:19:33.033122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.225708+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:19:32.225708+0000","last_peered":"2024-09-12T07:19:32.225708+0000","last_clean":"2024-09-12T07:19:32.225708+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:19:32.225708+0000","last_undegraded":"2024-09-12T07:19:32.225708+0000","last_fullsized":"2024-09-12T07:19:32.225708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:33.032725+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:19:33.032725+0000","last_peered":"2024-09-12T07:19:33.032725+0000","last_clean":"2024-09-12T07:19:33.032725+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:19:33.032725+0000","last_undegraded":"2024-09-12T07:19:33.032725+0000","last_fullsized":"2024-09-12T07:19:33.032725+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":806912,"data_stored":661406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":667238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:19:38.407 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T07:19:38.407 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T07:19:38.408 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-12T07:19:38.408 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-12T07:19:38.743 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:19:38.743 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T07:19:38.755 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-12T07:19:38.756 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-12T07:19:38.767 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-12T07:19:38.767 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing', 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}} 2024-09-12T07:19:38.767 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-12T07:19:38.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-12T07:19:39.083 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-12T07:19:39.083 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T07:19:39.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:19:39.380 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:19:39.381 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:19:39.392 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-12T07:19:37.492504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235796,"kb_used_data":2904,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738924,"statfs":{"total":773094113280,"available":772852658176,"internally_reserved":0,"allocated":2973696,"data_stored":1784070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.281985"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.474512+0000","last_change":"2024-09-12T07:19:32.474512+0000","last_active":"2024-09-12T07:19:32.474512+0000","last_peered":"2024-09-12T07:19:32.474512+0000","last_clean":"2024-09-12T07:19:32.474512+0000","last_became_active":"2024-09-12T07:19:30.219990+0000","last_became_peered":"2024-09-12T07:19:30.219990+0000","last_unstale":"2024-09-12T07:19:32.474512+0000","last_undegraded":"2024-09-12T07:19:32.474512+0000","last_fullsized":"2024-09-12T07:19:32.474512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:32.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046339800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.228066+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:19:32.228066+0000","last_peered":"2024-09-12T07:19:32.228066+0000","last_clean":"2024-09-12T07:19:32.228066+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:19:32.228066+0000","last_undegraded":"2024-09-12T07:19:32.228066+0000","last_fullsized":"2024-09-12T07:19:32.228066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:33.033060+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:19:33.033060+0000","last_peered":"2024-09-12T07:19:33.033060+0000","last_clean":"2024-09-12T07:19:33.033060+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:19:33.033060+0000","last_undegraded":"2024-09-12T07:19:33.033060+0000","last_fullsized":"2024-09-12T07:19:33.033060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.228114+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:19:32.228114+0000","last_peered":"2024-09-12T07:19:32.228114+0000","last_clean":"2024-09-12T07:19:32.228114+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:19:32.228114+0000","last_undegraded":"2024-09-12T07:19:32.228114+0000","last_fullsized":"2024-09-12T07:19:32.228114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.227049+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:19:32.227049+0000","last_peered":"2024-09-12T07:19:32.227049+0000","last_clean":"2024-09-12T07:19:32.227049+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:19:32.227049+0000","last_undegraded":"2024-09-12T07:19:32.227049+0000","last_fullsized":"2024-09-12T07:19:32.227049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.754632+0000","last_change":"2024-09-12T07:19:32.754826+0000","last_active":"2024-09-12T07:19:32.754632+0000","last_peered":"2024-09-12T07:19:32.754632+0000","last_clean":"2024-09-12T07:19:32.754632+0000","last_became_active":"2024-09-12T07:19:30.217328+0000","last_became_peered":"2024-09-12T07:19:30.217328+0000","last_unstale":"2024-09-12T07:19:32.754632+0000","last_undegraded":"2024-09-12T07:19:32.754632+0000","last_fullsized":"2024-09-12T07:19:32.754632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:57.208282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:33.033122+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:19:33.033122+0000","last_peered":"2024-09-12T07:19:33.033122+0000","last_clean":"2024-09-12T07:19:33.033122+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:19:33.033122+0000","last_undegraded":"2024-09-12T07:19:33.033122+0000","last_fullsized":"2024-09-12T07:19:33.033122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.225708+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:19:32.225708+0000","last_peered":"2024-09-12T07:19:32.225708+0000","last_clean":"2024-09-12T07:19:32.225708+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:19:32.225708+0000","last_undegraded":"2024-09-12T07:19:32.225708+0000","last_fullsized":"2024-09-12T07:19:32.225708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:33.032725+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:19:33.032725+0000","last_peered":"2024-09-12T07:19:33.032725+0000","last_clean":"2024-09-12T07:19:33.032725+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:19:33.032725+0000","last_undegraded":"2024-09-12T07:19:33.032725+0000","last_fullsized":"2024-09-12T07:19:33.032725+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":806912,"data_stored":661406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":667238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:19:39.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:19:39.692 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:19:39.692 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:19:39.704 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-12T07:19:39.492944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235804,"kb_used_data":2928,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738916,"statfs":{"total":773094113280,"available":772852649984,"internally_reserved":0,"allocated":2998272,"data_stored":1795734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.276734"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.474512+0000","last_change":"2024-09-12T07:19:32.474512+0000","last_active":"2024-09-12T07:19:32.474512+0000","last_peered":"2024-09-12T07:19:32.474512+0000","last_clean":"2024-09-12T07:19:32.474512+0000","last_became_active":"2024-09-12T07:19:30.219990+0000","last_became_peered":"2024-09-12T07:19:30.219990+0000","last_unstale":"2024-09-12T07:19:32.474512+0000","last_undegraded":"2024-09-12T07:19:32.474512+0000","last_fullsized":"2024-09-12T07:19:32.474512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:32.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046339800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.228066+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:19:32.228066+0000","last_peered":"2024-09-12T07:19:32.228066+0000","last_clean":"2024-09-12T07:19:32.228066+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:19:32.228066+0000","last_undegraded":"2024-09-12T07:19:32.228066+0000","last_fullsized":"2024-09-12T07:19:32.228066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:33.033060+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:19:33.033060+0000","last_peered":"2024-09-12T07:19:33.033060+0000","last_clean":"2024-09-12T07:19:33.033060+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:19:33.033060+0000","last_undegraded":"2024-09-12T07:19:33.033060+0000","last_fullsized":"2024-09-12T07:19:33.033060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.228114+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:19:32.228114+0000","last_peered":"2024-09-12T07:19:32.228114+0000","last_clean":"2024-09-12T07:19:32.228114+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:19:32.228114+0000","last_undegraded":"2024-09-12T07:19:32.228114+0000","last_fullsized":"2024-09-12T07:19:32.228114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.227049+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:19:32.227049+0000","last_peered":"2024-09-12T07:19:32.227049+0000","last_clean":"2024-09-12T07:19:32.227049+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:19:32.227049+0000","last_undegraded":"2024-09-12T07:19:32.227049+0000","last_fullsized":"2024-09-12T07:19:32.227049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.754632+0000","last_change":"2024-09-12T07:19:32.754826+0000","last_active":"2024-09-12T07:19:32.754632+0000","last_peered":"2024-09-12T07:19:32.754632+0000","last_clean":"2024-09-12T07:19:32.754632+0000","last_became_active":"2024-09-12T07:19:30.217328+0000","last_became_peered":"2024-09-12T07:19:30.217328+0000","last_unstale":"2024-09-12T07:19:32.754632+0000","last_undegraded":"2024-09-12T07:19:32.754632+0000","last_fullsized":"2024-09-12T07:19:32.754632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:57.208282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:33.033122+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:19:33.033122+0000","last_peered":"2024-09-12T07:19:33.033122+0000","last_clean":"2024-09-12T07:19:33.033122+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:19:33.033122+0000","last_undegraded":"2024-09-12T07:19:33.033122+0000","last_fullsized":"2024-09-12T07:19:33.033122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:32.225708+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:19:32.225708+0000","last_peered":"2024-09-12T07:19:32.225708+0000","last_clean":"2024-09-12T07:19:32.225708+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:19:32.225708+0000","last_undegraded":"2024-09-12T07:19:32.225708+0000","last_fullsized":"2024-09-12T07:19:32.225708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-12T07:19:33.032725+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:19:33.032725+0000","last_peered":"2024-09-12T07:19:33.032725+0000","last_clean":"2024-09-12T07:19:33.032725+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:19:33.032725+0000","last_undegraded":"2024-09-12T07:19:33.032725+0000","last_fullsized":"2024-09-12T07:19:33.032725+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":667238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":667238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:19:39.705 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T07:19:39.705 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-12T07:19:40.003 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 22 2024-09-12T07:19:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:fsid 8412ed58-ac5e-4549-b523-1a58cf9f9429 2024-09-12T07:19:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-12T07:19:10.328057+0000 2024-09-12T07:19:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-12T07:19:32.215704+0000 2024-09-12T07:19:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T07:19:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:crush_version 5 2024-09-12T07:19:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:full_ratio 0.95 2024-09-12T07:19:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:backfillfull_ratio 0.9 2024-09-12T07:19:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:nearfull_ratio 0.85 2024-09-12T07:19:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:require_min_compat_client luminous 2024-09-12T07:19:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:min_compat_client jewel 2024-09-12T07:19:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:require_osd_release squid 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout:stretch_mode_enabled false 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: removed_snaps_queue [2~1] 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout:max_osd 8 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6816/4081287959,v1:172.21.15.46:6817/4081287959] [v2:172.21.15.46:6818/4081287959,v1:172.21.15.46:6819/4081287959] exists,up 710c25c3-92d3-4b51-a7fc-670879faa69c 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6824/3980488213,v1:172.21.15.46:6825/3980488213] [v2:172.21.15.46:6826/3980488213,v1:172.21.15.46:6827/3980488213] exists,up 04479191-c52e-4b60-aa0d-3e911f869062 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6800/656802655,v1:172.21.15.46:6801/656802655] [v2:172.21.15.46:6802/656802655,v1:172.21.15.46:6803/656802655] exists,up ba673e51-96aa-424b-a4df-26432d633ebe 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6808/3848260327,v1:172.21.15.46:6809/3848260327] [v2:172.21.15.46:6810/3848260327,v1:172.21.15.46:6811/3848260327] exists,up c516e59c-834c-4fb0-9ff8-bcc80b21d28a 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6816/3464256744,v1:172.21.15.64:6817/3464256744] [v2:172.21.15.64:6818/3464256744,v1:172.21.15.64:6819/3464256744] exists,up 450a70cd-ebb2-48ed-a766-184f58e7032e 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6824/1789414693,v1:172.21.15.64:6825/1789414693] [v2:172.21.15.64:6826/1789414693,v1:172.21.15.64:6827/1789414693] exists,up a4efa333-bf57-45e7-a281-1d3bc9d2e82b 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6800/2576774646,v1:172.21.15.64:6801/2576774646] [v2:172.21.15.64:6802/2576774646,v1:172.21.15.64:6804/2576774646] exists,up bcaad352-f9ee-426c-bcbf-39c8d031aae0 2024-09-12T07:19:40.005 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6803/368918691,v1:172.21.15.64:6805/368918691] [v2:172.21.15.64:6807/368918691,v1:172.21.15.64:6809/368918691] exists,up 97e5ef36-58a4-4667-9f9e-b8ced5904d69 2024-09-12T07:19:40.017 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6816/4081287959,v1:172.21.15.46:6817/4081287959] [v2:172.21.15.46:6818/4081287959,v1:172.21.15.46:6819/4081287959] exists,up 710c25c3-92d3-4b51-a7fc-670879faa69c', 'osd.1 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6824/3980488213,v1:172.21.15.46:6825/3980488213] [v2:172.21.15.46:6826/3980488213,v1:172.21.15.46:6827/3980488213] exists,up 04479191-c52e-4b60-aa0d-3e911f869062', 'osd.2 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6800/656802655,v1:172.21.15.46:6801/656802655] [v2:172.21.15.46:6802/656802655,v1:172.21.15.46:6803/656802655] exists,up ba673e51-96aa-424b-a4df-26432d633ebe', 'osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6808/3848260327,v1:172.21.15.46:6809/3848260327] [v2:172.21.15.46:6810/3848260327,v1:172.21.15.46:6811/3848260327] exists,up c516e59c-834c-4fb0-9ff8-bcc80b21d28a', 'osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6816/3464256744,v1:172.21.15.64:6817/3464256744] [v2:172.21.15.64:6818/3464256744,v1:172.21.15.64:6819/3464256744] exists,up 450a70cd-ebb2-48ed-a766-184f58e7032e', 'osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6824/1789414693,v1:172.21.15.64:6825/1789414693] [v2:172.21.15.64:6826/1789414693,v1:172.21.15.64:6827/1789414693] exists,up a4efa333-bf57-45e7-a281-1d3bc9d2e82b', 'osd.6 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6800/2576774646,v1:172.21.15.64:6801/2576774646] [v2:172.21.15.64:6802/2576774646,v1:172.21.15.64:6804/2576774646] exists,up bcaad352-f9ee-426c-bcbf-39c8d031aae0', 'osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6803/368918691,v1:172.21.15.64:6805/368918691] [v2:172.21.15.64:6807/368918691,v1:172.21.15.64:6809/368918691] exists,up 97e5ef36-58a4-4667-9f9e-b8ced5904d69'] 2024-09-12T07:19:40.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-12T07:19:40.121 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mon.a", 2024-09-12T07:19:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster": "ceph", 2024-09-12T07:19:40.121 INFO:teuthology.orchestra.run.smithi046.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "admin_socket_mode": "", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_client_required": "cephx, none", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_cluster_required": "cephx", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_debug": "true", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_service_required": "cephx", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_supported": "cephx", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_aio": "true", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-12T07:19:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_aio_poll_ms": "250", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_aio_reap_max": "16", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_async_discard_threads": "0", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_block_size": "4096", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_debug_aio": "false", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_debug_inflight_ios": "false", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_enable_discard": "false", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_flock_retry": "3", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_inject_crash": "0", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-12T07:19:40.123 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_ioring": "false", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_ioring_hipri": "false", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bdev_type": "", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_alloc_size": "1048576", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_allocator": "hybrid", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_buffered_io": "true", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_check_for_zeros": "false", 2024-09-12T07:19:40.124 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_compact_log_sync": "false", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_max_log_runway": "4194304", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_max_prefetch": "1048576", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_min_flush_size": "524288", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_min_log_runway": "1048576", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_replay_recovery": "false", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-12T07:19:40.125 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_sync_write": "false", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_allocation_from_file": "true", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_allocator": "hybrid", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_async_db_compaction": "true", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_bdev_label_multi": "true", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-12T07:19:40.126 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_block_create": "true", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_block_db_create": "false", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_block_db_path": "", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_block_db_size": "0", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_block_path": "", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_block_preallocate_file": "false", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_block_size": "107374182400", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_block_wal_create": "false", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_block_wal_path": "", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_block_wal_size": "100663296", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_bluefs": "true", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-12T07:19:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_autotune": "true", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-12T07:19:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_size": "0", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_type": "2q", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_clone_cow": "true", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-12T07:19:40.129 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_compression_mode": "none", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_csum_type": "crc32c", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_freelist": "false", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-12T07:19:40.130 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_small_allocations": "0", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_default_buffered_read": "true", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_default_buffered_write": "false", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-12T07:19:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fail_eio": "false", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_on_mount": "false", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_on_umount": "false", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-12T07:19:40.132 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_ignore_data_csum": "false", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_log_op_age": "5.000000", 2024-09-12T07:19:40.133 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_max_alloc_size": "0", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_max_blob_size": "0", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_max_deferred_txc": "32", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_min_alloc_size": "0", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_nid_prealloc": "1024", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-12T07:19:40.134 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_retry_disk_reads": "3", 2024-09-12T07:19:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_rocksdb_cf": "true", 2024-09-12T07:19:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-12T07:19:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-12T07:19:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-12T07:19:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-12T07:19:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-12T07:19:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-12T07:19:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-12T07:19:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-12T07:19:40.135 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_spdk_mem": "512", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_tracing": "false", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-12T07:19:40.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-12T07:19:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-12T07:19:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-12T07:19:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-12T07:19:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_write_v2": "false", 2024-09-12T07:19:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_write_v2_random": "false", 2024-09-12T07:19:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_zero_block_detection": "false", 2024-09-12T07:19:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-12T07:19:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-12T07:19:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-12T07:19:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-12T07:19:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-12T07:19:40.137 INFO:teuthology.orchestra.run.smithi046.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephx_cluster_require_signatures": "false", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephx_cluster_require_version": "2", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephx_require_signatures": "false", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephx_require_version": "2", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephx_service_require_signatures": "false", 2024-09-12T07:19:40.138 INFO:teuthology.orchestra.run.smithi046.stdout: "cephx_service_require_version": "2", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "cephx_sign_messages": "true", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "chdir": "", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_acl_type": "", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_asio_thread_count": "2", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_cache_mid": "0.750000", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_cache_size": "16384", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_caps_release_delay": "5", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_check_pool_perm": "true", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_debug_force_sync_read": "false", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_debug_getattr_caps": "false", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_debug_inject_features": "", 2024-09-12T07:19:40.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_debug_inject_tick_delay": "0", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_die_on_failed_remount": "false", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_dirsize_rbytes": "true", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_force_lazyio": "false", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_fs": "", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_inject_release_failure": "false", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_max_inline_size": "4096", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_mds_namespace": "", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_metadata": "", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_mount_gid": "-1", 2024-09-12T07:19:40.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_mount_timeout": "300", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_mount_uid": "-1", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_mountpoint": "/", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_notify_timeout": "10", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_oc": "true", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_oc_max_dirty": "104857600", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_oc_max_objects": "1000", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_oc_size": "209715200", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_oc_target_dirty": "8388608", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_permissions": "true", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_quota": "true", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_quota_df": "true", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_readahead_max_bytes": "0", 2024-09-12T07:19:40.141 INFO:teuthology.orchestra.run.smithi046.stdout: "client_readahead_max_periods": "4", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_readahead_min": "131072", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_reconnect_stale": "false", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_shutdown_timeout": "30", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_snapdir": ".snap", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_tick_interval": "1", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_trace": "", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_try_dentry_invalidate": "false", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_use_faked_inos": "false", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_use_random_mds": "false", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "clog_to_graylog": "false", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "clog_to_graylog_port": "12201", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "clog_to_monitors": "default=true", 2024-09-12T07:19:40.142 INFO:teuthology.orchestra.run.smithi046.stdout: "clog_to_syslog": "false", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "clog_to_syslog_level": "info", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "-", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_network": "", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_network_interface": "", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "compressor_zlib_isal": "false", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "compressor_zlib_level": "5", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "compressor_zlib_winsize": "-15", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "compressor_zstd_level": "1", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "crimson_alien_op_num_threads": "6", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-12T07:19:40.143 INFO:teuthology.orchestra.run.smithi046.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "crimson_osd_stat_interval": "0", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "crimson_seastar_cpu_cores": "", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "crimson_seastar_num_threads": "0", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_location": "", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_location_hook": "", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_location_hook_timeout": "10", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "daemonize": "false", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "daos_pool": "tank", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-12T07:19:40.144 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_alienstore": "0/5", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_allow_any_pool_priority": "false", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_asok": "1/5", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_asok_assert_abort": "false", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_asserts_on_shutdown": "true", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_auth": "20/20", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_bdev": "1/3", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_bluefs": "1/5", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_bluestore": "1/5", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_buffer": "0/1", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_ceph_dedup": "0/5", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_ceph_exporter": "1/5", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_cephfs_mirror": "0/5", 2024-09-12T07:19:40.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_cephsqlite": "0/5", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_client": "0/5", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_compressor": "1/5", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_context": "0/1", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_crimson_interrupt": "0/5", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_crush": "1/1", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_crypto": "1/5", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_cyanstore": "0/5", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_deliberately_leak_memory": "false", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_disable_randomized_ping": "false", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_dpdk": "1/5", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_eventtrace": "1/5", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_filer": "0/1", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_filestore": "1/3", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_finisher": "1/1", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_fuse": "1/5", 2024-09-12T07:19:40.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_heartbeat_testing_span": "0", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_heartbeatmap": "1/5", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_javaclient": "1/5", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_journal": "1/3", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_journaler": "0/5", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_kstore": "1/5", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_lockdep": "0/1", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_mclock": "1/5", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_mds": "1/5", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_mds_balancer": "1/5", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_mds_locker": "1/5", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_mds_log": "1/5", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_mds_log_expire": "1/5", 2024-09-12T07:19:40.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_mds_migrator": "1/5", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_mds_quiesce": "3/5", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_memstore": "1/5", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_mgr": "2/5", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_mgrc": "1/5", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_mon": "20/20", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_monc": "0/10", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_ms": "1/1", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_none": "0/5", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_objclass": "0/5", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_objectcacher": "0/5", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_objecter": "0/1", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_optracker": "0/5", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_osd": "1/5", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_paxos": "20/20", 2024-09-12T07:19:40.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_perfcounter": "1/5", 2024-09-12T07:19:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_prioritycache": "1/5", 2024-09-12T07:19:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rados": "0/5", 2024-09-12T07:19:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rbd": "0/5", 2024-09-12T07:19:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rbd_mirror": "0/5", 2024-09-12T07:19:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rbd_pwl": "0/5", 2024-09-12T07:19:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rbd_replay": "0/5", 2024-09-12T07:19:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_refs": "0/0", 2024-09-12T07:19:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_reserver": "1/1", 2024-09-12T07:19:40.149 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rgw": "1/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rgw_access": "1/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rgw_datacache": "1/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rgw_dbstore": "1/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rgw_flight": "1/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rgw_notification": "1/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rgw_sync": "1/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_rocksdb": "4/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore": "0/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_backref": "0/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_cache": "0/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_cleaner": "0/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_device": "0/5", 2024-09-12T07:19:40.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_epm": "0/5", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_journal": "0/5", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_lba": "0/5", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_odata": "0/5", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_omap": "0/5", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_onode": "0/5", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_t": "0/5", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_seastore_tm": "0/5", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_striper": "0/1", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_test": "0/5", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_throttle": "1/1", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_timer": "0/1", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_tp": "0/5", 2024-09-12T07:19:40.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug_trace": "1/5", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "device_failure_prediction_mode": "none", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "err_to_graylog": "false", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "err_to_journald": "false", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "err_to_stderr": "true", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "err_to_syslog": "false", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event_tracing": "false", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "exporter_addr": "0.0.0.0", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "exporter_cert_file": "", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "exporter_http_port": "9926", 2024-09-12T07:19:40.152 INFO:teuthology.orchestra.run.smithi046.stdout: "exporter_key_file": "", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "exporter_prio_limit": "5", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "exporter_sort_metrics": "true", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "exporter_stats_period": "5", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "fake_statfs_for_testing": "0", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "fatal_signal_handlers": "true", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "filer_max_purge_ops": "10", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "filer_max_truncate_ops": "128", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_apply_finisher_threads": "1", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_blackhole": "false", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_btrfs_clone_range": "true", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_btrfs_snap": "true", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_caller_concurrency": "10", 2024-09-12T07:19:40.153 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_collect_device_partition_information": "true", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_commit_timeout": "600.000000", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_debug_inject_read_err": "false", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_debug_omap_check": "false", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_debug_verify_split": "false", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_dump_file": "", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_fadvise": "true", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_fail_eio": "true", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_fd_cache_shards": "16", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_fd_cache_size": "128", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_fiemap": "false", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_fiemap_threshold": "4096", 2024-09-12T07:19:40.154 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_inject_stall": "0", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_journal_parallel": "false", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_journal_trailing": "false", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_journal_writeahead": "false", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_kill_at": "0", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_inline_xattrs": "0", 2024-09-12T07:19:40.155 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_xattr_value_size": "0", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_merge_threshold": "-10", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_odsync_write": "false", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_omap_backend": "rocksdb", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_omap_backend_path": "", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-12T07:19:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_op_thread_timeout": "60", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_op_threads": "2", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_punch_hole": "false", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-12T07:19:40.157 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_queue_max_ops": "50", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_seek_data_hole": "false", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_sloppy_crc": "false", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_splice": "false", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_split_multiple": "2", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_split_rand_factor": "20", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_update_to": "1000", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-12T07:19:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_enable": "true", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_xfs_extsize": "false", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "filestore_zfs_snap": "false", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "fio_dir": "/tmp/fio", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "8412ed58-ac5e-4549-b523-1a58cf9f9429", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_allow_other": "true", 2024-09-12T07:19:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_atomic_o_trunc": "true", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_big_writes": "true", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_debug": "false", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_default_permissions": "false", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_disable_pagecache": "false", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_max_write": "0", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_multithreaded": "true", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_require_active_mds": "true", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_set_user_groups": "true", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_splice_move": "true", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_splice_read": "true", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_splice_write": "true", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-12T07:19:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "fuse_use_invalidate_cb": "true", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "gss_target_name": "ceph", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_file": "", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_inject_failure": "0", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_interval": "5", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "host": "smithi046", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-12T07:19:40.161 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "inject_early_sigterm": "false", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "jaeger_agent_port": "6799", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "jaeger_tracing_enable": "false", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_aio": "true", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_align_min_size": "65536", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_block_align": "true", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_block_size": "4096", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_dio": "true", 2024-09-12T07:19:40.162 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_discard": "false", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_force_aio": "false", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_ignore_corruption": "false", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_max_write_bytes": "10485760", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_max_write_entries": "100", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_replay_from": "0", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_write_header_frequency": "0", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journal_zero_on_create": "false", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journaler_prefetch_periods": "10", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journaler_prezero_periods": "5", 2024-09-12T07:19:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: "journaler_write_head_interval": "15", 2024-09-12T07:19:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "", 2024-09-12T07:19:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "keyfile": "", 2024-09-12T07:19:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-12T07:19:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "kstore_backend": "rocksdb", 2024-09-12T07:19:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "kstore_default_stripe_size": "65536", 2024-09-12T07:19:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "kstore_fsck_on_mount": "false", 2024-09-12T07:19:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-12T07:19:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "kstore_max_bytes": "67108864", 2024-09-12T07:19:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "kstore_max_ops": "512", 2024-09-12T07:19:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "kstore_nid_prealloc": "1024", 2024-09-12T07:19:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "kstore_onode_map_size": "1024", 2024-09-12T07:19:40.164 INFO:teuthology.orchestra.run.smithi046.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "kstore_sync_submit_transaction": "false", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "kstore_sync_transaction": "false", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "librados_thread_count": "2", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "lockdep": "false", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "lockdep_force_backtrace": "false", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_coarse_timestamps": "true", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_flush_on_exit": "false", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_graylog_host": "127.0.0.1", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_graylog_port": "12201", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_max_new": "1000", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_max_recent": "10000", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_stderr_prefix": "", 2024-09-12T07:19:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_stop_at_utilization": "0.970000", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": "true", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_graylog": "false", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_journald": "false", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_stderr": "false", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_syslog": "false", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "max_rotating_auth_attempts": "10", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_action_on_write_error": "1", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_alternate_name_max": "8192", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_asio_thread_count": "2", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_export_pin": "true", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_fragment_dirs": "true", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-12T07:19:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_fragment_interval": "5", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_interval": "10", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_max": "-1", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_max_until": "-1", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_merge_size": "50", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_midchunk": "0.300000", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_min_start": "0.200000", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_minchunk": "0.001000", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_mode": "0", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_need_max": "1.200000", 2024-09-12T07:19:40.167 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_need_min": "0.800000", 2024-09-12T07:19:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_overload_epochs": "2", 2024-09-12T07:19:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-12T07:19:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-12T07:19:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_split_bits": "3", 2024-09-12T07:19:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-12T07:19:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_split_size": "10000", 2024-09-12T07:19:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-12T07:19:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_target_decay": "10.000000", 2024-09-12T07:19:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-12T07:19:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_beacon_grace": "15.000000", 2024-09-12T07:19:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_beacon_interval": "4.000000", 2024-09-12T07:19:40.168 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache_mid": "0.700000", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache_quiesce_delay": "0", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache_release_free_interval": "10", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache_reservation": "0.050000", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache_trim_interval": "1", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache_trim_threshold": "262144", 2024-09-12T07:19:40.169 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_client_prealloc_inos": "1000", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_connect_bootstrapping": "false", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_damage_table_max_entries": "10000", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_debug_auth_pins": "false", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_debug_frag": "false", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_debug_scatterstat": "false", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_debug_subtrees": "false", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_decay_halflife": "5.000000", 2024-09-12T07:19:40.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_default_dir_hash": "2", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_defer_session_stale": "true", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_deny_all_reconnect": "false", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_dir_keys_per_op": "16384", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_dir_max_commit_size": "10", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_dir_max_entries": "0", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_dir_prefetch": "true", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_dump_cache_on_map": "false", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_early_reply": "true", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_enable_op_tracker": "true", 2024-09-12T07:19:40.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_enforce_unique_name": "true", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_export_ephemeral_random": "true", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_health_summarize_threshold": "10", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-12T07:19:40.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_inject_health_dummy": "false", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_inject_migrator_session_race": "false", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_join_fs": "", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_journal_format": "1", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_create_at": "0", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_dirfrag_at": "0", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_export_at": "0", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_import_at": "0", 2024-09-12T07:19:40.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_journal_at": "0", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_journal_expire_at": "0", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_journal_replay_at": "0", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_link_at": "0", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_mdstable_at": "0", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_openc_at": "0", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_rename_at": "0", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_kill_shutdown_at": "0", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_event_large_threshold": "524288", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_events_per_segment": "1024", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_max_events": "-1", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_max_segments": "128", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_pause": "false", 2024-09-12T07:19:40.174 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_segment_size": "0", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_trim_threshold": "128", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log_warn_factor": "2.000000", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_max_caps_per_client": "1048576", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_max_completed_flushes": "100000", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_max_completed_requests": "100000", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_max_export_size": "20971520", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_max_file_recover": "32", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_max_purge_files": "64", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_max_purge_ops": "8192", 2024-09-12T07:19:40.175 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-12T07:19:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-12T07:19:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_max_snaps_per_dir": "100", 2024-09-12T07:19:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_metrics_update_interval": "2", 2024-09-12T07:19:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_min_caps_per_client": "100", 2024-09-12T07:19:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_min_caps_working_set": "10000", 2024-09-12T07:19:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-12T07:19:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_numa_node": "-1", 2024-09-12T07:19:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-12T07:19:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_op_complaint_time": "30.000000", 2024-09-12T07:19:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_op_history_duration": "600", 2024-09-12T07:19:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_op_history_size": "20", 2024-09-12T07:19:40.176 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_op_history_slow_op_size": "20", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_op_log_threshold": "5", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_ping_grace": "15", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_ping_interval": "5", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_recall_max_caps": "30000", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_recall_warning_threshold": "262144", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_replay_interval": "1.000000", 2024-09-12T07:19:40.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_root_ino_gid": "0", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_root_ino_uid": "0", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_scrub_stats_review_period": "1", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-12T07:19:40.178 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_shutdown_check": "0", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_skip_ino": "0", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_snap_max_uid": "4294967294", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_snap_min_uid": "0", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_snap_rstat": "false", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_standby_replay_damaged": "false", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_symlink_recovery": "true", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_thrash_exports": "0", 2024-09-12T07:19:40.179 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_thrash_fragments": "0", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_tick_interval": "5.000000", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_valgrind_exit": "false", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_verify_backtrace": "1", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_verify_scatter": "false", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_wipe_sessions": "false", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool_debug": "false", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "memstore_device_bytes": "1073741824", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "memstore_page_set": "false", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "memstore_page_size": "65536", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_client_bytes": "134217728", 2024-09-12T07:19:40.180 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_client_messages": "512", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_disabled_modules": "", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_enable_op_tracker": "true", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_max_pg_creating": "1024", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_max_pg_num_change": "128", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_mds_bytes": "134217728", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_mds_messages": "128", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-12T07:19:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_mon_bytes": "134217728", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_mon_messages": "128", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_op_history_duration": "600", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_op_history_size": "20", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_op_log_threshold": "5", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_osd_bytes": "536870912", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_osd_messages": "8192", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_pool": "true", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_standby_modules": "true", 2024-09-12T07:19:40.182 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_stats_period": "5", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_stats_threshold": "5", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_tick_period": "2", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_allow_pool_delete": "true", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_allow_pool_size_one": "true", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_auth_validate_all_caps": "true", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_client_bytes": "104857600", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_client_directed_command_retry": "5", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-12T07:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_client_hunt_parallel": "3", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_client_log_interval": "1.000000", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_client_ping_interval": "10.000000", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_client_target_rank": "-1", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_cluster_log_level": "debug", 2024-09-12T07:19:40.184 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_cluster_log_to_file": "true", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_cluster_log_to_journald": "false", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_compact_on_bootstrap": "false", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_compact_on_start": "false", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_compact_on_trim": "true", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-12T07:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_cpu_threads": "4", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_crush_min_required_version": "hammer", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_daemon_bytes": "419430400", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_data_avail_crit": "5", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_data_avail_warn": "5", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_data_size_warn": "16106127360", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_debug_dump_json": "false", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_debug_dump_transactions": "false", 2024-09-12T07:19:40.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_debug_extra_checks": "false", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_debug_no_require_reef": "false", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_debug_no_require_squid": "false", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_down_mkfs_grace": "120", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_down_uptime_grace": "60", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_election_default_strategy": "1", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_election_timeout": "5.000000", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_elector_ping_divisor": "2", 2024-09-12T07:19:40.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_enable_op_tracker": "true", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_fake_pool_delete": "false", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_force_quorum_join": "false", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_globalid_prealloc": "10000", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_health_detail_to_clog": "false", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_health_log_update_period": "5", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_health_max_detail": "50", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_health_to_clog": "true", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_health_to_clog_interval": "600", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_host": "172.21.15.46,[v2:172.21.15.46:3301,v1:172.21.15.46:6790],172.21.15.64", 2024-09-12T07:19:40.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_host_override": "", 2024-09-12T07:19:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_initial_members": "", 2024-09-12T07:19:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-12T07:19:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-12T07:19:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-12T07:19:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-12T07:19:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-12T07:19:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_lease": "5.000000", 2024-09-12T07:19:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-12T07:19:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-12T07:19:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_log_full_interval": "50", 2024-09-12T07:19:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_log_max": "10000", 2024-09-12T07:19:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_log_max_summary": "50", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_max_log_epochs": "500", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_max_osd": "10000", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_max_pg_per_osd": "10000", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_max_pool_pg_num": "65536", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_mds_force_trim_to": "0", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_mds_skip_sanity": "false", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_memory_autotune": "true", 2024-09-12T07:19:40.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_memory_target": "2147483648", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_mgr_beacon_grace": "30", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_mgr_digest_period": "5", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_mgr_inactive_grace": "60", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_min_osdmap_epochs": "500", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_op_complaint_time": "30", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_op_history_duration": "600", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_op_history_size": "20", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_op_history_slow_op_size": "20", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-12T07:19:40.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_op_log_threshold": "5", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_auto_mark_in": "false", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_cache_size": "500", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_down_out_interval": "600", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-12T07:19:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_force_trim_to": "0", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_min_down_reporters": "2", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-12T07:19:40.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_report_timeout": "900", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-12T07:19:40.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_pg_stuck_threshold": "60", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_probe_timeout": "2.000000", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_reweight_max_change": "0.050000", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_reweight_max_osds": "4", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-12T07:19:40.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_scrub_interval": "86400", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_scrub_max_keys": "100", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_scrub_timeout": "300", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_session_timeout": "300", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_smart_report_timeout": "5", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_stat_smooth_intervals": "6", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_stretch_pool_min_size": "2", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_stretch_pool_size": "4", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_sync_debug": "false", 2024-09-12T07:19:40.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_sync_provider_kill_at": "0", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_sync_requester_kill_at": "0", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_sync_timeout": "60.000000", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_target_pg_per_osd": "100", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_tick_interval": "5", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_timecheck_interval": "300.000000", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_use_min_delay_socket": "false", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_older_version_delay": "604800", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-12T07:19:40.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_misplaced": "false", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_older_version": "true", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-12T07:19:40.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": "", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "motr_tracing_enabled": "false", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_op_threads": "3", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-12T07:19:40.199 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_cm": "false", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_device_name": "", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_dscp": "96", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_local_gid": "", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_port_num": "1", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_sl": "3", 2024-09-12T07:19:40.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_support_srq": "true", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_rdma_type": "ib", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_async_reap_threshold": "5", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_bind_before_connect": "false", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_bind_ipv4": "true", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_bind_ipv6": "false", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_bind_msgr1": "true", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_bind_msgr2": "true", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_bind_port_max": "7568", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_bind_port_min": "6800", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_bind_retry_count": "3", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_bind_retry_delay": "5", 2024-09-12T07:19:40.201 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_blackhole_client": "false", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_blackhole_mds": "false", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_blackhole_mgr": "false", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_blackhole_mon": "false", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_blackhole_osd": "false", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_client_mode": "crc secure", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_cluster_mode": "crc secure", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_cluster_type": "", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_compress_secure": "false", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_connection_idle_timeout": "900", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_connection_ready_timeout": "10", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_crc_data": "true", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_crc_header": "true", 2024-09-12T07:19:40.202 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_die_on_bad_msg": "false", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_die_on_bug": "true", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_die_on_old_message": "true", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_die_on_skipped_message": "false", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_coremask": "0xF", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_enable_tso": "true", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_hugepages": "", 2024-09-12T07:19:40.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_lro": "true", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_memory_channel": "4", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_pmd": "", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_port_id": "0", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_dump_on_send": "false", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_initial_backoff": "0.200000", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_inject_delay_max": "1.000000", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-12T07:19:40.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_inject_delay_type": "", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_inject_network_congestion": "0", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_inject_socket_failures": "5000", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_learn_addr_from_peer": "true", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_max_accept_failures": "4", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_max_backoff": "15.000000", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_mon_client_mode": "secure crc", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_mon_service_mode": "secure crc", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_osd_compress_min_size": "1024", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_osd_compress_mode": "none", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-12T07:19:40.205 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_pq_min_cost": "65536", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_public_type": "", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_service_mode": "crc secure", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_tcp_listen_backlog": "512", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_tcp_nodelay": "true", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_tcp_rcvbuf": "0", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "ms_type": "async+posix", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "no_config_file": "false", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-12T07:19:40.206 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter_completion_locks_per_session": "32", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter_inflight_ops": "1024", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter_tick_interval": "5.000000", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter_timeout": "10.000000", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "objectstore_blackhole": "false", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "openssl_engine_opts": "", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_agent_delay_time": "5.000000", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_agent_hist_halflife": "1000", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_agent_max_low_ops": "2", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_agent_max_ops": "4", 2024-09-12T07:19:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_agent_slop": "0.020000", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_asio_thread_count": "2", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_async_recovery_min_cost": "100", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_backfill_scan_max": "512", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_backfill_scan_min": "64", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_backoff_on_degraded": "false", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_backoff_on_peering": "false", 2024-09-12T07:19:40.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_backoff_on_unfound": "true", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_beacon_report_interval": "300", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bench_duration": "30", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bench_max_block_size": "67108864", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_blkin_trace_all": "false", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_check_for_log_corruption": "false", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-12T07:19:40.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-12T07:19:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-12T07:19:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-12T07:19:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_class_update_on_start": "true", 2024-09-12T07:19:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_client_message_cap": "256", 2024-09-12T07:19:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_client_message_size_cap": "524288000", 2024-09-12T07:19:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_client_op_priority": "63", 2024-09-12T07:19:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_client_watch_timeout": "30", 2024-09-12T07:19:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_command_max_records": "256", 2024-09-12T07:19:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_compact_on_start": "false", 2024-09-12T07:19:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-12T07:19:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-12T07:19:40.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_crush_update_on_start": "true", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_crush_update_weight_set": "true", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_feed_pullee": "-1", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_misdirected_ops": "false", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_no_acting_change": "false", 2024-09-12T07:19:40.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_no_purge_strays": "false", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_op_order": "false", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_shutdown": "false", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_trim_objects": "false", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_verify_snaps": "false", 2024-09-12T07:19:40.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_deep_scrub_keys": "1024", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_deep_scrub_stride": "524288", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_default_notify_timeout": "30", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_delete_sleep": "0.000000", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-12T07:19:40.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-12T07:19:40.214 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_discard_disconnected_ops": "true", 2024-09-12T07:19:40.214 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_ec_partial_reads": "true", 2024-09-12T07:19:40.214 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_enable_op_tracker": "true", 2024-09-12T07:19:40.214 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-12T07:19:40.214 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-12T07:19:40.214 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-12T07:19:40.214 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-12T07:19:40.214 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_fast_info": "true", 2024-09-12T07:19:40.214 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_fast_shutdown": "true", 2024-09-12T07:19:40.214 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-12T07:19:40.214 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_function_tracing": "false", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_heartbeat_grace": "20", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_heartbeat_interval": "6", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_heartbeat_min_peers": "10", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_heartbeat_min_size": "2000", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_heartbeat_stale": "600", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_hit_set_max_size": "100000", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_hit_set_min_size": "1000", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-12T07:19:40.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-12T07:19:40.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-12T07:19:40.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-12T07:19:40.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-12T07:19:40.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-12T07:19:40.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_journal_size": "5120", 2024-09-12T07:19:40.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_kill_backfill_at": "0", 2024-09-12T07:19:40.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-12T07:19:40.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_map_cache_size": "50", 2024-09-12T07:19:40.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_map_dedup": "true", 2024-09-12T07:19:40.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_map_message_max": "40", 2024-09-12T07:19:40.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-12T07:19:40.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_map_share_max_epochs": "40", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_attr_name_len": "100", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_attr_size": "0", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_backfills": "1", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_markdown_count": "5", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_markdown_period": "600", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_object_name_len": "2048", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_object_namespace_len": "256", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_object_size": "134217728", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_pg_blocked_by": "16", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_pg_log_entries": "10000", 2024-09-12T07:19:40.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_pgls": "1024", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_push_cost": "8388608", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_push_objects": "10", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_scrubs": "3", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_trimming_pgs": "2", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_write_op_reply_len": "64", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_max_write_size": "90", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-12T07:19:40.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_profile": "balanced", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-12T07:19:40.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_memory_base": "805306368", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_memory_cache_min": "134217728", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_memory_target": "4294967296", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_memory_target_autotune": "false", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_min_pg_log_entries": "250", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_min_recovery_priority": "0", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mon_report_interval": "5", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-12T07:19:40.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_num_cache_shards": "32", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_num_op_tracker_shard": "32", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_numa_auto_affinity": "true", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_numa_node": "-1", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_numa_prefer_iface": "true", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_objecter_finishers": "1", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_objectstore": "bluestore", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_objectstore_fuse": "false", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_objectstore_tracing": "false", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_complaint_time": "30.000000", 2024-09-12T07:19:40.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_history_duration": "600", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_history_size": "20", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_history_slow_op_size": "20", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_log_threshold": "5", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_num_shards": "0", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_num_shards_hdd": "5", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_num_shards_ssd": "8", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_pq_min_cost": "65536", 2024-09-12T07:19:40.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_queue_cut_off": "high", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_op_thread_timeout": "15", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_open_classes_on_start": "true", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_os_flags": "0", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_peering_op_priority": "255", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pg_delete_cost": "1048576", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pg_delete_priority": "5", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pg_log_trim_max": "10000", 2024-09-12T07:19:40.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pg_log_trim_min": "100", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_crimson": "false", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-12T07:19:40.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_flags": "0", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_min_size": "0", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_pg_num": "32", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_pgp_num": "0", 2024-09-12T07:19:40.225 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-12T07:19:40.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_read_ratio": "70", 2024-09-12T07:19:40.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_size": "2", 2024-09-12T07:19:40.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_default_type": "replicated", 2024-09-12T07:19:40.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-12T07:19:40.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-12T07:19:40.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_push_per_object_cost": "1000", 2024-09-12T07:19:40.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-12T07:19:40.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recover_clone_overlap": "true", 2024-09-12T07:19:40.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-12T07:19:40.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_cost": "20971520", 2024-09-12T07:19:40.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-12T07:19:40.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_max_active": "0", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_max_single_start": "1", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_op_priority": "3", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_priority": "5", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_sleep": "0.000000", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-12T07:19:40.227 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_repair_during_recovery": "false", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_requested_scrub_priority": "120", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_auto_repair": "false", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_begin_hour": "0", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_begin_week_day": "0", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_chunk_max": "25", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_chunk_min": "5", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_cost": "52428800", 2024-09-12T07:19:40.228 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-12T07:19:40.229 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_during_recovery": "false", 2024-09-12T07:19:40.229 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_end_hour": "0", 2024-09-12T07:19:40.229 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_end_week_day": "0", 2024-09-12T07:19:40.229 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_event_cost": "4096", 2024-09-12T07:19:40.229 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-12T07:19:40.232 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-12T07:19:40.232 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_invalid_stats": "true", 2024-09-12T07:19:40.232 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-12T07:19:40.232 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-12T07:19:40.232 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_max_preemptions": "5", 2024-09-12T07:19:40.232 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-12T07:19:40.232 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_priority": "5", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_retry_delay": "30", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_retry_trimming": "10", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_scrub_sleep": "0.000000", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_skip_data_digest": "false", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_smart_report_timeout": "5", 2024-09-12T07:19:40.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_snap_trim_cost": "1048576", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_snap_trim_priority": "5", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_target_transaction_size": "30", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-12T07:19:40.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-12T07:19:40.235 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-12T07:19:40.235 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-12T07:19:40.235 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-12T07:19:40.235 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-12T07:19:40.235 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-12T07:19:40.235 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-12T07:19:40.235 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_tracing": "false", 2024-09-12T07:19:40.235 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_use_stale_snap": "false", 2024-09-12T07:19:40.235 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-12T07:19:40.235 INFO:teuthology.orchestra.run.smithi046.stdout: "osdc_blkin_trace_all": "false", 2024-09-12T07:19:40.235 INFO:teuthology.orchestra.run.smithi046.stdout: "paxos_kill_at": "0", 2024-09-12T07:19:40.235 INFO:teuthology.orchestra.run.smithi046.stdout: "paxos_max_join_drift": "10", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "paxos_min": "500", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "paxos_min_wait": "0.050000", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "paxos_propose_interval": "1.000000", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "paxos_service_trim_max": "500", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "paxos_service_trim_min": "250", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "paxos_stash_full_interval": "25", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "paxos_trim_max": "500", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "paxos_trim_min": "250", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "perf": "true", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-12T07:19:40.236 INFO:teuthology.orchestra.run.smithi046.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "-", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrv": "", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "public_bind_addr": "-", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "public_network": "", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "public_network_interface": "", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "qat_compressor_busy_polling": "false", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "qat_compressor_enabled": "false", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "qat_compressor_session_max_number": "256", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "rados_mon_op_timeout": "0", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "rados_osd_op_timeout": "0", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "rados_tracing": "false", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_atime_update_interval": "60", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-12T07:19:40.237 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_balance_parent_reads": "false", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_balance_snap_reads": "false", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_blkin_trace_all": "false", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_cache": "true", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_cache_policy": "writearound", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_cache_size": "33554432", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-12T07:19:40.238 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_clone_copy_on_read": "false", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_compression_hint": "none", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_concurrent_management_ops": "10", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_default_clone_format": "auto", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_default_data_pool": "", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_default_features": "61", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_default_format": "2", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_default_map_options": "", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_default_order": "22", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_default_pool": "rbd", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_default_stripe_count": "0", 2024-09-12T07:19:40.239 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_default_stripe_unit": "0", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_enable_alloc_hint": "true", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_io_scheduler": "simple", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-12T07:19:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-12T07:19:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-12T07:19:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_journal_order": "24", 2024-09-12T07:19:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_journal_pool": "", 2024-09-12T07:19:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_journal_splay_width": "4", 2024-09-12T07:19:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_localize_parent_reads": "false", 2024-09-12T07:19:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_localize_snap_reads": "false", 2024-09-12T07:19:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-12T07:19:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-12T07:19:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-12T07:19:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-12T07:19:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-12T07:19:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-12T07:19:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-12T07:19:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-12T07:19:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-12T07:19:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-12T07:19:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-12T07:19:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-12T07:19:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-12T07:19:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-12T07:19:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-12T07:19:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-12T07:19:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-12T07:19:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-12T07:19:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_mtime_update_interval": "60", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_non_blocking_aio": "true", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_op_thread_timeout": "60", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_op_threads": "1", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_parent_cache_enabled": "false", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_plugins": "", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_bps_burst": "0", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_bps_limit": "0", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_exclude_ops": "0", 2024-09-12T07:19:40.244 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_iops_burst": "0", 2024-09-12T07:19:40.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-12T07:19:40.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_iops_limit": "0", 2024-09-12T07:19:40.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-12T07:19:40.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-12T07:19:40.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-12T07:19:40.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-12T07:19:40.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-12T07:19:40.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-12T07:19:40.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-12T07:19:40.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-12T07:19:40.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-12T07:19:40.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_read_from_replica_policy": "default", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_skip_partial_discard": "true", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_tracing": "false", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_validate_names": "true", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_validate_pool": "true", 2024-09-12T07:19:40.246 INFO:teuthology.orchestra.run.smithi046.stdout: "restapi_base_url": "", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "restapi_log_level": "", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_acl_grants_max_num": "100", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_admin_entry": "admin", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_asio_assert_yielding": "false", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_backend_store": "rados", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_barbican_url": "", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_beast_enable_async": "true", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_bucket_counters_cache": "false", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-12T07:19:40.247 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_cache_enabled": "true", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_cache_expiry_interval": "900", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_cache_lru_size": "25000", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_config_store": "rados", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_content_length_compat": "false", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_copy_obj_progress": "true", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-12T07:19:40.248 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_cors_rules_max_num": "100", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_cross_domain_policy": "", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_kmip_addr": "", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_kmip_password": "", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_kmip_username": "", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_require_ssl": "true", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-12T07:19:40.249 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-12T07:19:40.250 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-12T07:19:40.250 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-12T07:19:40.250 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-12T07:19:40.250 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-12T07:19:40.250 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-12T07:19:40.250 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-12T07:19:40.250 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-12T07:19:40.250 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-12T07:19:40.250 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_vault_addr": "", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_vault_auth": "token", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_vault_namespace": "", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_vault_prefix": "", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-12T07:19:40.251 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_vault_token_file": "", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_curl_buffersize": "524288", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_curl_low_speed_time": "300", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-12T07:19:40.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_data_log_changes_size": "1000", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_data_log_num_shards": "128", 2024-09-12T07:19:40.253 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_data_log_window": "30", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-12T07:19:40.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_disable_s3select": "false", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-12T07:19:40.255 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dns_name": "", 2024-09-12T07:19:40.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dns_s3website_name": "", 2024-09-12T07:19:40.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dynamic_resharding": "true", 2024-09-12T07:19:40.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-12T07:19:40.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-12T07:19:40.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-12T07:19:40.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_enable_gc_threads": "true", 2024-09-12T07:19:40.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_enable_lc_threads": "true", 2024-09-12T07:19:40.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_enable_mdsearch": "true", 2024-09-12T07:19:40.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_enable_ops_log": "false", 2024-09-12T07:19:40.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_enable_quota_threads": "true", 2024-09-12T07:19:40.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_enable_static_website": "false", 2024-09-12T07:19:40.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_enable_usage_log": "false", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_enforce_swift_acls": "true", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_exit_timeout_secs": "120", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_expose_bucket": "false", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_extended_http_attrs": "", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_filter": "none", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_frontends": "beast port=7480", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_gc_max_deferred": "50", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_gc_max_objs": "32", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-12T07:19:40.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_gc_processor_period": "3600", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_init_timeout": "300", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_kafka_connection_idle": "300", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-12T07:19:40.258 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_admin_domain": "", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_admin_password": "", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_admin_password_path": "", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_admin_project": "", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_admin_tenant": "", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_admin_token": "", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_admin_token_path": "", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_admin_user": "", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_barbican_domain": "", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_barbican_password": "", 2024-09-12T07:19:40.259 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_barbican_project": "", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_barbican_user": "", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_url": "", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_lc_debug_interval": "-1", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_lc_lock_max_time": "90", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_lc_max_objs": "32", 2024-09-12T07:19:40.260 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_lc_max_rules": "1000", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_lc_max_worker": "3", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_lc_thread_delay": "0", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_ldap_dnattr": "uid", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_ldap_searchfilter": "", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_ldap_uri": "", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-12T07:19:40.261 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_log_http_headers": "", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_log_object_name_utc": "false", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_attr_name_len": "0", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_attr_size": "0", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_chunk_size": "4194304", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-12T07:19:40.262 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_listing_results": "5000", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_notify_retries": "10", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_put_param_size": "1048576", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_put_size": "5368709120", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_max_slo_entries": "1000", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_md_log_max_shards": "64", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_mp_lock_max_time": "600", 2024-09-12T07:19:40.263 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_max_gc": "300", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-12T07:19:40.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_num_async_rados_threads": "32", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_num_control_oids": "8", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_numa_node": "-1", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_objexp_chunk_size": "100", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_objexp_gc_interval": "600", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-12T07:19:40.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_op_thread_timeout": "600", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_op_tracing": "false", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_opa_token": "", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_opa_url": "", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_opa_verify_ssl": "true", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_ops_log_rados": "false", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_ops_log_socket_path": "", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_period_push_interval": "2.000000", 2024-09-12T07:19:40.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_posix_cache_lanes": "3", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_posix_cache_partitions": "3", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_print_continue": "true", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-12T07:19:40.267 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_rados_tracing": "false", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_realm": "", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_realm_id": "", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_region": "", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_request_uri": "", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_reshard_batch_size": "64", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-12T07:19:40.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_reshard_max_aio": "128", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_reshard_num_logs": "16", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_reshard_thread_interval": "600", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_resolve_cname": "false", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_run_sync_thread": "true", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-12T07:19:40.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_scheduler_type": "throttler", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_script_uri": "", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_service_provider_name": "", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sts_client_id": "", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sts_client_secret": "", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sts_entry": "sts", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sts_key": "sts", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sts_min_session_duration": "900", 2024-09-12T07:19:40.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sts_token_introspection_url": "", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_swift_account_in_url": "false", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_swift_auth_entry": "auth", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_swift_auth_url": "", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_swift_custom_header": "", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_swift_need_stats": "true", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_swift_tenant_name": "", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_swift_token_expiration": "86400", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_swift_url": "", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_swift_url_prefix": "swift", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-12T07:19:40.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sync_lease_period": "120", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_thread_pool_size": "512", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-12T07:19:40.272 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_torrent_comment": "", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_torrent_createby": "", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_torrent_encoding": "", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_torrent_flag": "false", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_torrent_origin": "", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_torrent_tracker": "", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_trust_forwarded_https": "false", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-12T07:19:40.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_usage_max_shards": "32", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_usage_max_user_shards": "1", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_use_opa_authz": "false", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_user_counters_cache": "false", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_user_max_buckets": "1000", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_user_policies_max_num": "100", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-12T07:19:40.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_user_unique_email": "true", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_verify_ssl": "true", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_zone": "", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_zone_id": "", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_zonegroup": "", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_zonegroup_id": "", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rgwlc_auto_session_clear": "true", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_block_size": "4096", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-12T07:19:40.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_cache_size": "536870912", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-12T07:19:40.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_index_type": "binary_search", 2024-09-12T07:19:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-12T07:19:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-12T07:19:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_partition_filters": "false", 2024-09-12T07:19:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_perf": "false", 2024-09-12T07:19:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-12T07:19:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-12T07:19:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-12T07:19:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: "run_dir": "/var/run/ceph", 2024-09-12T07:19:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_block_create": "true", 2024-09-12T07:19:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_cache_lru_size": "67108864", 2024-09-12T07:19:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-12T07:19:40.277 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_default_max_object_size": "16777216", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_device_size": "53687091200", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_full_integrity_check": "false", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_journal_batch_capacity": "16", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_main_device_type": "SSD", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-12T07:19:40.278 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "seastore_segment_size": "67108864", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "set_keepcaps": "false", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "setgroup": "", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "setuser": "", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "setuser_match_path": "", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "thp": "false", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "threadpool_default_timeout": "60", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "throttler_perf_counter": "true", 2024-09-12T07:19:40.279 INFO:teuthology.orchestra.run.smithi046.stdout: "tmp_dir": "/tmp", 2024-09-12T07:19:40.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-12T07:19:40.280 INFO:teuthology.orchestra.run.smithi046.stdout: "uadk_compressor_enabled": "false", 2024-09-12T07:19:40.280 INFO:teuthology.orchestra.run.smithi046.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-12T07:19:40.280 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:40.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-12T07:19:40.399 INFO:teuthology.orchestra.run.smithi046.stdout:mon.a: {} 2024-09-12T07:19:40.400 INFO:teuthology.orchestra.run.smithi046.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-12T07:19:40.409 INFO:teuthology.orchestra.run.smithi046.stdout:mon.b: {} 2024-09-12T07:19:40.409 INFO:teuthology.orchestra.run.smithi046.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-12T07:19:40.418 INFO:teuthology.orchestra.run.smithi046.stdout:mon.c: {} 2024-09-12T07:19:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-12T07:19:40.429 DEBUG:teuthology.orchestra.run.smithi046:> type ceph-objectstore-tool 2024-09-12T07:19:40.444 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-12T07:19:40.444 DEBUG:teuthology.orchestra.run.smithi064:> type ceph-objectstore-tool 2024-09-12T07:19:40.463 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-12T07:19:40.464 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-12T07:19:40.473 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-12T07:19:40.473 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-12T07:19:40.473 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:19:40.473 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:19:40.474 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-12T07:19:40.474 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.051850 2024-09-12T07:19:40.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.05185003819128853 2024-09-12T07:19:40.474 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-12T07:19:40.474 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-12T07:19:40.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:19:40.475 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-12T07:19:40.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:19:40.475 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-12T07:19:40.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:19:40.476 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-12T07:19:40.504 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:40.506+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:40.531 INFO:teuthology.orchestra.run.smithi046.stdout:mapped writes DISABLED 2024-09-12T07:19:40.531 INFO:teuthology.orchestra.run.smithi046.stdout:mapped reads DISABLED 2024-09-12T07:19:40.531 INFO:teuthology.orchestra.run.smithi046.stdout:Seed set to 4076 2024-09-12T07:19:40.576 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:40.577+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:40.642 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:40.643 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:40.643 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:19:40.643 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:40.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:19:40.673 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:19:40.673 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:19:40.676 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:40.678+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:40.691 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:19:40.691 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:19:40.708 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:19:40.709 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:19:40.726 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:19:40.727 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:19:40.745 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:19:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:19:40.763 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:19:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:19:40.764 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:40.764 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:40.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:19:40.777 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:40.779+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:40.786 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:19:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:19:40.804 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:19:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:19:40.878 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:40.880+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:40.896 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:40.896 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:40.904 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:19:40.979 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:40.980+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:41.011 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:41.011 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:41.021 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:19:41.079 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:41.081+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:41.128 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:41.128 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:41.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:19:41.180 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:41.181+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:41.280 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:41.282+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:41.284 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:41.284 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:41.293 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:19:41.381 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:41.383+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:41.436 INFO:teuthology.orchestra.run.smithi046.stderr:set osd.4 primary-affinity to 0.05185 (833982) 2024-09-12T07:19:41.440 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:19:41.446 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:41.446 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:41.458 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:19:41.499 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:41.501+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:41.583 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:41.585+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:41.606 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:41.606 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:41.615 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:19:41.684 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:41.686+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:41.722 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:41.722 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:41.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:19:41.814 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:41.816+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:41.878 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:41.878 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:41.886 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:19:41.887 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:41.888+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:41.987 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:41.989+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:42.033 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:42.034 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:42.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:19:42.088 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:42.089+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:42.150 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:42.150 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:42.158 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:19:42.203 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:42.205+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:42.277 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:42.280 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:42.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:19:42.289 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:42.291+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:42.390 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:42.392+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:42.435 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:42.435 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:42.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:19:42.491 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:42.493+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:42.556 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:42.557 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:42.565 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:19:42.609 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:42.611+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:42.671 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:42.672 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:42.680 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:19:42.693 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:42.695+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:42.794 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:42.796+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:42.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:19:42.826 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:42.826 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:42.834 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:19:42.895 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:42.896+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:42.941 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:42.941 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:42.949 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:19:42.951 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:19:42.951 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:19:42.969 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:19:42.969 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:19:42.985 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:19:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:19:42.995 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:42.997+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:43.001 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:19:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:19:43.018 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:19:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:19:43.035 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:19:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:19:43.052 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:19:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:19:43.056 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:43.057 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:43.057 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:19:43.057 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:43.065 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:19:43.068 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:19:43.069 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:19:43.120 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:43.122+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:43.198 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:43.199+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:43.212 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:43.212 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:43.212 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:19:43.212 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:19:43.212 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:43.221 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:19:43.305 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:43.307+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:43.367 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:43.368 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:19:43.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:19:43.368 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [] 2024-09-12T07:19:43.368 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:43.376 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:19:43.417 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:43.419+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:43.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T07:19:43.503 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:43.505+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:43.524 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:43.524 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:43.524 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:19:43.525 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:43.533 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:19:43.601 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:43.603+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:43.639 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:43.639 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:43.639 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:19:43.639 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:19:43.639 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:43.647 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:19:43.702 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:43.703+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:43.756 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:43.756 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:19:43.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:19:43.756 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:19:43.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.756 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.756 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.304755+0000", 2024-09-12T07:19:43.756 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.450924229000002, 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0051170749999999996, 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 43 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304755+0000", 2024-09-12T07:19:43.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304755+0000", 2024-09-12T07:19:43.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304756+0000", 2024-09-12T07:19:43.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.02e-06 2024-09-12T07:19:43.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304762+0000", 2024-09-12T07:19:43.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4139999999999999e-06 2024-09-12T07:19:43.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304782+0000", 2024-09-12T07:19:43.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9570000000000001e-05 2024-09-12T07:19:43.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304799+0000", 2024-09-12T07:19:43.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7132000000000001e-05 2024-09-12T07:19:43.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.307263+0000", 2024-09-12T07:19:43.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024647699999999998 2024-09-12T07:19:43.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.307314+0000", 2024-09-12T07:19:43.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0716e-05 2024-09-12T07:19:43.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.307395+0000", 2024-09-12T07:19:43.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1100000000000006e-05 2024-09-12T07:19:43.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.309096+0000", 2024-09-12T07:19:43.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017005869999999999 2024-09-12T07:19:43.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.309765+0000", 2024-09-12T07:19:43.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00066909999999999995 2024-09-12T07:19:43.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.309816+0000", 2024-09-12T07:19:43.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1326999999999998e-05 2024-09-12T07:19:43.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.309872+0000", 2024-09-12T07:19:43.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5339e-05 2024-09-12T07:19:43.767 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.767 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.767 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.767 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.767 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.304825+0000", 2024-09-12T07:19:43.767 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.450854137, 2024-09-12T07:19:43.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057401880000000002, 2024-09-12T07:19:43.767 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.767 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 44 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304825+0000", 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304825+0000", 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304825+0000", 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6599999999999995e-07 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304893+0000", 2024-09-12T07:19:43.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7552999999999995e-05 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304904+0000", 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0628999999999999e-05 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304908+0000", 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8080000000000004e-06 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.307787+0000", 2024-09-12T07:19:43.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028791350000000001 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.307859+0000", 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1453999999999997e-05 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.307960+0000", 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100832 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.310453+0000", 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024930960000000002 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.310468+0000", 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5233e-05 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.310523+0000", 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4753000000000002e-05 2024-09-12T07:19:43.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.310565+0000", 2024-09-12T07:19:43.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1928999999999998e-05 2024-09-12T07:19:43.773 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.773 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.773 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.773 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.304924+0000", 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.450754591999999, 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070038289999999996, 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 45 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304924+0000", 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304924+0000", 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304925+0000", 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9599999999999999e-07 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304929+0000", 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5879999999999999e-06 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304936+0000", 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2870000000000001e-06 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304940+0000", 2024-09-12T07:19:43.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6739999999999996e-06 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.308406+0000", 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00346599 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.308502+0000", 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6207999999999998e-05 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.308649+0000", 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00014693900000000001 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.310594+0000", 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019443100000000001 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.311825+0000", 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001231062 2024-09-12T07:19:43.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.311885+0000", 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9771000000000001e-05 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.311928+0000", 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3404e-05 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.304950+0000", 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.450728902000002, 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070741780000000004, 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.780 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304950+0000", 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304950+0000", 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304950+0000", 2024-09-12T07:19:43.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:19:43.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304964+0000", 2024-09-12T07:19:43.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3611999999999999e-05 2024-09-12T07:19:43.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304973+0000", 2024-09-12T07:19:43.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7110000000000001e-06 2024-09-12T07:19:43.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304977+0000", 2024-09-12T07:19:43.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9740000000000004e-06 2024-09-12T07:19:43.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.309152+0000", 2024-09-12T07:19:43.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0041759509999999998 2024-09-12T07:19:43.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.309246+0000", 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3407999999999997e-05 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.309407+0000", 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016110000000000001 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.310649+0000", 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001242274 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.311938+0000", 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001288525 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.311986+0000", 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8239999999999999e-05 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.312024+0000", 2024-09-12T07:19:43.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.799e-05 2024-09-12T07:19:43.786 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.786 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.786 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.786 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.786 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.304984+0000", 2024-09-12T07:19:43.786 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.450694520999999, 2024-09-12T07:19:43.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0071475849999999997, 2024-09-12T07:19:43.786 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.786 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 47 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304984+0000", 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304984+0000", 2024-09-12T07:19:43.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.304985+0000", 2024-09-12T07:19:43.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:19:43.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305031+0000", 2024-09-12T07:19:43.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6205000000000002e-05 2024-09-12T07:19:43.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305044+0000", 2024-09-12T07:19:43.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2706e-05 2024-09-12T07:19:43.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305050+0000", 2024-09-12T07:19:43.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7020000000000004e-06 2024-09-12T07:19:43.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.309906+0000", 2024-09-12T07:19:43.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0048556600000000004 2024-09-12T07:19:43.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.309957+0000", 2024-09-12T07:19:43.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1456000000000003e-05 2024-09-12T07:19:43.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.310053+0000", 2024-09-12T07:19:43.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5427000000000006e-05 2024-09-12T07:19:43.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.311811+0000", 2024-09-12T07:19:43.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001758045 2024-09-12T07:19:43.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.312032+0000", 2024-09-12T07:19:43.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000220977 2024-09-12T07:19:43.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.312078+0000", 2024-09-12T07:19:43.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5843000000000001e-05 2024-09-12T07:19:43.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.312132+0000", 2024-09-12T07:19:43.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4147000000000002e-05 2024-09-12T07:19:43.795 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.796 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.796 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.796 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.796 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.305065+0000", 2024-09-12T07:19:43.796 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.450613393000001, 2024-09-12T07:19:43.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.007212614, 2024-09-12T07:19:43.796 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305065+0000", 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305065+0000", 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305066+0000", 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3399999999999999e-07 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305071+0000", 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6500000000000004e-06 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305078+0000", 2024-09-12T07:19:43.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6149999999999999e-06 2024-09-12T07:19:43.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305092+0000", 2024-09-12T07:19:43.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3704e-05 2024-09-12T07:19:43.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.310676+0000", 2024-09-12T07:19:43.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055840890000000004 2024-09-12T07:19:43.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.310723+0000", 2024-09-12T07:19:43.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.676e-05 2024-09-12T07:19:43.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.310824+0000", 2024-09-12T07:19:43.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000101569 2024-09-12T07:19:43.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.312142+0000", 2024-09-12T07:19:43.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001317494 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.312197+0000", 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5198999999999998e-05 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.312232+0000", 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4462999999999999e-05 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.312278+0000", 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6437000000000003e-05 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.305119+0000", 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.450559715000001, 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0093293620000000008, 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.803 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 49 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305119+0000", 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305119+0000", 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305120+0000", 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8599999999999997e-07 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305125+0000", 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1880000000000001e-06 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305134+0000", 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1390000000000001e-06 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305140+0000", 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9399999999999999e-06 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.311146+0000", 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0060059969999999999 2024-09-12T07:19:43.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.311254+0000", 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107503 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.311400+0000", 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000146069 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.313174+0000", 2024-09-12T07:19:43.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001774236 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.314348+0000", 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001174406 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.314405+0000", 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6341000000000001e-05 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.314448+0000", 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3757000000000001e-05 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.305158+0000", 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.450520872999999, 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0093851390000000007, 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.809 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 50 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305158+0000", 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305158+0000", 2024-09-12T07:19:43.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305158+0000", 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305222+0000", 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3689000000000006e-05 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305234+0000", 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2313999999999999e-05 2024-09-12T07:19:43.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305241+0000", 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.917e-06 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.312329+0000", 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070876949999999998 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.312419+0000", 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0255999999999998e-05 2024-09-12T07:19:43.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.312570+0000", 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015082800000000001 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.314334+0000", 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017639769999999999 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.314458+0000", 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012382199999999999 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.314503+0000", 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5358000000000002e-05 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.314543+0000", 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9814999999999997e-05 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.814 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.305256+0000", 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.450422795000001, 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010339875, 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 51 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305256+0000", 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305256+0000", 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305257+0000", 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0100000000000005e-07 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305262+0000", 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0919999999999997e-06 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305278+0000", 2024-09-12T07:19:43.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5984000000000001e-05 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305284+0000", 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7739999999999999e-06 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.313219+0000", 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0079343160000000003 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.313330+0000", 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000110841 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.313523+0000", 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000193785 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.314574+0000", 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0010507400000000001 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.315500+0000", 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00092595100000000005 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.315549+0000", 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8817999999999999e-05 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.315596+0000", 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6972999999999999e-05 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.305295+0000", 2024-09-12T07:19:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.450383536, 2024-09-12T07:19:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010663376, 2024-09-12T07:19:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.822 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.822 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.822 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.822 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 52 2024-09-12T07:19:43.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.822 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305295+0000", 2024-09-12T07:19:43.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305295+0000", 2024-09-12T07:19:43.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305296+0000", 2024-09-12T07:19:43.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7800000000000002e-07 2024-09-12T07:19:43.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305299+0000", 2024-09-12T07:19:43.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.41e-06 2024-09-12T07:19:43.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305305+0000", 2024-09-12T07:19:43.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4189999999999999e-06 2024-09-12T07:19:43.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.305309+0000", 2024-09-12T07:19:43.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.444e-06 2024-09-12T07:19:43.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.313991+0000", 2024-09-12T07:19:43.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0086824459999999999 2024-09-12T07:19:43.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.314044+0000", 2024-09-12T07:19:43.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2836e-05 2024-09-12T07:19:43.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.314124+0000", 2024-09-12T07:19:43.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0086999999999994e-05 2024-09-12T07:19:43.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.314897+0000", 2024-09-12T07:19:43.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00077275499999999997 2024-09-12T07:19:43.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.315859+0000", 2024-09-12T07:19:43.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000961878 2024-09-12T07:19:43.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.315918+0000", 2024-09-12T07:19:43.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8752000000000003e-05 2024-09-12T07:19:43.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.315959+0000", 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0970999999999998e-05 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.525296+0000", 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.230383152999998, 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0048781190000000002, 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.831 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.832 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.832 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.832 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 69 2024-09-12T07:19:43.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.832 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525296+0000", 2024-09-12T07:19:43.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525296+0000", 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525296+0000", 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9800000000000004e-07 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525333+0000", 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6996000000000003e-05 2024-09-12T07:19:43.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525349+0000", 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6112000000000001e-05 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525356+0000", 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9430000000000003e-06 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.528385+0000", 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030292069999999999 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.528453+0000", 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8141999999999999e-05 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.528592+0000", 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013802899999999999 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.529912+0000", 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0013200340000000001 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530063+0000", 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000151934 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530112+0000", 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8733999999999998e-05 2024-09-12T07:19:43.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530174+0000", 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1489999999999996e-05 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.525396+0000", 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.230282760000001, 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053484709999999996, 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.838 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.838 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.838 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.838 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.838 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 70 2024-09-12T07:19:43.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.838 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525396+0000", 2024-09-12T07:19:43.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525396+0000", 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525397+0000", 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4099999999999999e-07 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525401+0000", 2024-09-12T07:19:43.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.792e-06 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525415+0000", 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3782e-05 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525424+0000", 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.799e-06 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.529201+0000", 2024-09-12T07:19:43.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0037773989999999999 2024-09-12T07:19:43.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.529289+0000", 2024-09-12T07:19:43.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7508000000000003e-05 2024-09-12T07:19:43.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.529429+0000", 2024-09-12T07:19:43.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000139834 2024-09-12T07:19:43.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530641+0000", 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0012121689999999999 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530655+0000", 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3971000000000001e-05 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530704+0000", 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9289000000000002e-05 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530744+0000", 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0386999999999998e-05 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.525436+0000", 2024-09-12T07:19:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.230242947000001, 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0073448489999999996, 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 71 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525436+0000", 2024-09-12T07:19:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525436+0000", 2024-09-12T07:19:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525436+0000", 2024-09-12T07:19:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8999999999999996e-07 2024-09-12T07:19:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525441+0000", 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.386e-06 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525452+0000", 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1404e-05 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525458+0000", 2024-09-12T07:19:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9479999999999996e-06 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530216+0000", 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0047581979999999999 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530265+0000", 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8965000000000003e-05 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530343+0000", 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7274000000000006e-05 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.532057+0000", 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017146799999999999 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.532687+0000", 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00063004000000000001 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.532735+0000", 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7700000000000001e-05 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.532781+0000", 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5664000000000002e-05 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.525469+0000", 2024-09-12T07:19:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.230209635000001, 2024-09-12T07:19:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0085499870000000002, 2024-09-12T07:19:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 72 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525469+0000", 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525469+0000", 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525470+0000", 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6700000000000003e-07 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525531+0000", 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1206999999999999e-05 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525539+0000", 2024-09-12T07:19:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5799999999999992e-06 2024-09-12T07:19:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525550+0000", 2024-09-12T07:19:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.083e-05 2024-09-12T07:19:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530791+0000", 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0052408840000000003 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530839+0000", 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8038000000000003e-05 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.530933+0000", 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4256e-05 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.533799+0000", 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028652339999999999 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.533923+0000", 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012391100000000001 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.533973+0000", 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9956000000000001e-05 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534019+0000", 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6523999999999997e-05 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.525565+0000", 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.230114001, 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0090199630000000006, 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 73 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525565+0000", 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525565+0000", 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525565+0000", 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1900000000000002e-07 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525571+0000", 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4049999999999999e-06 2024-09-12T07:19:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525588+0000", 2024-09-12T07:19:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7285000000000001e-05 2024-09-12T07:19:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525599+0000", 2024-09-12T07:19:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0954999999999999e-05 2024-09-12T07:19:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.531425+0000", 2024-09-12T07:19:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005825556 2024-09-12T07:19:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.531511+0000", 2024-09-12T07:19:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6605000000000002e-05 2024-09-12T07:19:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.531650+0000", 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000138632 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.533844+0000", 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021943549999999998 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534489+0000", 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00064511699999999996 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534539+0000", 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0005e-05 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534585+0000", 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5328999999999999e-05 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.525615+0000", 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.230064013, 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0090563589999999999, 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 74 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525615+0000", 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525615+0000", 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525615+0000", 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4300000000000003e-07 2024-09-12T07:19:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525620+0000", 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9779999999999996e-06 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525630+0000", 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1679999999999993e-06 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525636+0000", 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0120000000000002e-06 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.532102+0000", 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0064662610000000001 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.532201+0000", 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9321999999999995e-05 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.532332+0000", 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013135299999999999 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.533908+0000", 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015753189999999999 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534591+0000", 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00068330599999999995 2024-09-12T07:19:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534634+0000", 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2542e-05 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534671+0000", 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7455e-05 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.525646+0000", 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.230032281, 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0091156129999999998, 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 75 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525646+0000", 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525646+0000", 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525647+0000", 2024-09-12T07:19:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.37e-07 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525713+0000", 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6272999999999994e-05 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525750+0000", 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.642e-05 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525758+0000", 2024-09-12T07:19:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.157e-06 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.532831+0000", 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070729060000000003 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.532923+0000", 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2267000000000003e-05 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.533000+0000", 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6789e-05 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534473+0000", 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0014727259999999999 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534678+0000", 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000205453 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534722+0000", 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3612999999999997e-05 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534762+0000", 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0472000000000001e-05 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.525774+0000", 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 18.229904593000001, 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0092638649999999996, 2024-09-12T07:19:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 76 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525774+0000", 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525774+0000", 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525775+0000", 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4300000000000003e-07 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525780+0000", 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525789+0000", 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.388e-06 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.525794+0000", 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6030000000000004e-06 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534068+0000", 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0082739489999999992 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534120+0000", 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1817999999999998e-05 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534196+0000", 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6397999999999996e-05 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534770+0000", 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00057415400000000001 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.534983+0000", 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000212924 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.535012+0000", 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8187999999999998e-05 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.535038+0000", 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6463000000000001e-05 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.804490+0000", 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 17.951188289000001, 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053278730000000003, 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 93 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.804490+0000", 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.804490+0000", 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.804491+0000", 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4600000000000001e-07 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.804599+0000", 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108377 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.804607+0000", 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7409999999999992e-06 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.804611+0000", 2024-09-12T07:19:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7170000000000002e-06 2024-09-12T07:19:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.807641+0000", 2024-09-12T07:19:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030304160000000002 2024-09-12T07:19:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.807685+0000", 2024-09-12T07:19:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3628999999999999e-05 2024-09-12T07:19:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.807758+0000", 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3467999999999995e-05 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.809384+0000", 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001626366 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.809768+0000", 2024-09-12T07:19:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000383975 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.809795+0000", 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6540999999999999e-05 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.809818+0000", 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3397e-05 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4181.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:25.804618+0000", 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 17.951061029999998, 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058002000000000001, 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4181", 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.64:0/4186274146", 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 94 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.804618+0000", 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.804618+0000", 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.804618+0000", 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:19:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.804620+0000", 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2230000000000001e-06 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.804624+0000", 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6030000000000001e-06 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.804626+0000", 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5399999999999998e-06 2024-09-12T07:19:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.809413+0000", 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004786691 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.809456+0000", 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2391000000000001e-05 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.809524+0000", 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8775999999999993e-05 2024-09-12T07:19:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.810046+0000", 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00052155799999999998 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.810330+0000", 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028392599999999999 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.810383+0000", 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3431999999999999e-05 2024-09-12T07:19:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:25.810418+0000", 2024-09-12T07:19:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4703999999999998e-05 2024-09-12T07:19:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:43.891 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:43.893 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:43.894+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:43.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:19:43.993 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:43.994+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:44.001 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:44.001 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:19:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:44.010 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:19:44.094 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:44.096+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:44.116 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:44.117 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:19:44.117 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:19:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:44.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:19:44.195 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:44.197+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:44.275 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:19:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:19:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:19:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:8 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'3, mlcod=17'1)", 2024-09-12T07:19:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.198872+0000", 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.077084227, 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00090540399999999995, 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.198872+0000", 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.198872+0000", 2024-09-12T07:19:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.198874+0000", 2024-09-12T07:19:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.716e-06 2024-09-12T07:19:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.198887+0000", 2024-09-12T07:19:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.257e-05 2024-09-12T07:19:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.198914+0000", 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7736e-05 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.198932+0000", 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7617000000000001e-05 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.199060+0000", 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012835900000000001 2024-09-12T07:19:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.199099+0000", 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9048999999999998e-05 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.199740+0000", 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000641016 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.199778+0000", 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7341e-05 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:14 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'5, mlcod=17'3)", 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.203704+0000", 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.072252591000002, 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00084885699999999995, 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.203704+0000", 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.203704+0000", 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.203706+0000", 2024-09-12T07:19:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8029999999999999e-06 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.203720+0000", 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3883000000000001e-05 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.203747+0000", 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7803000000000001e-05 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.203760+0000", 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.294e-05 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.203906+0000", 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014534199999999999 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.203959+0000", 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3433000000000002e-05 2024-09-12T07:19:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.204513+0000", 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00055357799999999999 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.204553+0000", 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0074999999999998e-05 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.210761+0000", 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.065195436, 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011487330000000001, 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210761+0000", 2024-09-12T07:19:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210761+0000", 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210762+0000", 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6000000000000002e-07 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210863+0000", 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100614 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210876+0000", 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3526999999999999e-05 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210888+0000", 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2140000000000001e-05 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210936+0000", 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7753000000000002e-05 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210959+0000", 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3394e-05 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211855+0000", 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00089539600000000002 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211910+0000", 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5049000000000001e-05 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.211373+0000", 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.064583218999999, 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001078702, 2024-09-12T07:19:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211373+0000", 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211373+0000", 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211374+0000", 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2600000000000002e-07 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211474+0000", 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9932999999999993e-05 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211485+0000", 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1029999999999999e-05 2024-09-12T07:19:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211495+0000", 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5829999999999996e-06 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211549+0000", 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4452e-05 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211567+0000", 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8476999999999999e-05 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212385+0000", 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081729700000000003 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212452+0000", 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7204000000000002e-05 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:22 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'12, mlcod=17'8)", 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.211865+0000", 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.064090993000001, 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099281999999999994, 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211865+0000", 2024-09-12T07:19:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211865+0000", 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211866+0000", 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0500000000000003e-07 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211973+0000", 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106456 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211985+0000", 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2286000000000001e-05 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211991+0000", 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-12T07:19:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212032+0000", 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0528000000000002e-05 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212047+0000", 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4625e-05 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212824+0000", 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00077776699999999995 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212858+0000", 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3973000000000002e-05 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:23 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'13, mlcod=17'8)", 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.212427+0000", 2024-09-12T07:19:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.063529119999998, 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000904243, 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212427+0000", 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212427+0000", 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212428+0000", 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7899999999999997e-07 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212482+0000", 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3919999999999999e-05 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212494+0000", 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1778e-05 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212500+0000", 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6239999999999999e-06 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212544+0000", 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4239000000000001e-05 2024-09-12T07:19:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212566+0000", 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2359000000000001e-05 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.213296+0000", 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00073030100000000004 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.213332+0000", 2024-09-12T07:19:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5142999999999999e-05 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:26 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'16, mlcod=17'14)", 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.214840+0000", 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.061116162000001, 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000854534, 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.214840+0000", 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.214840+0000", 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.214841+0000", 2024-09-12T07:19:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0500000000000003e-07 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.214857+0000", 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6494000000000002e-05 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.214867+0000", 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7890000000000006e-06 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.214872+0000", 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5519999999999998e-06 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.214906+0000", 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4220000000000001e-05 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.214919+0000", 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2771e-05 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.215662+0000", 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00074372700000000002 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.215695+0000", 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2376000000000003e-05 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:34 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'20, mlcod=17'18)", 2024-09-12T07:19:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.301699+0000", 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.974257338000001, 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001000302, 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301699+0000", 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301699+0000", 2024-09-12T07:19:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301713+0000", 2024-09-12T07:19:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3517e-05 2024-09-12T07:19:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301767+0000", 2024-09-12T07:19:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3902e-05 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301811+0000", 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4523e-05 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301829+0000", 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8029999999999998e-05 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301993+0000", 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016428600000000001 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.302017+0000", 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3436999999999999e-05 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.302667+0000", 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00064969599999999995 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.302699+0000", 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2910999999999997e-05 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:39 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'21, mlcod=17'19)", 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.304727+0000", 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.971229090000001, 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079889200000000005, 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.304727+0000", 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.304727+0000", 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.304728+0000", 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0200000000000001e-07 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.304745+0000", 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6701000000000002e-05 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.304757+0000", 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2488e-05 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.304769+0000", 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1747e-05 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.304843+0000", 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3993999999999997e-05 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.304862+0000", 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8749999999999998e-05 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305470+0000", 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00060793399999999997 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305526+0000", 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6475999999999999e-05 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:41 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'23, mlcod=17'19)", 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.305735+0000", 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.970221980000002, 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010266030000000001, 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305735+0000", 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305735+0000", 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305735+0000", 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0200000000000002e-07 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305814+0000", 2024-09-12T07:19:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8709999999999997e-05 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305823+0000", 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1589999999999999e-06 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305827+0000", 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1899999999999997e-06 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305863+0000", 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5648000000000002e-05 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305878+0000", 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5611e-05 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.306716+0000", 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00083814200000000005 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.306761+0000", 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4641000000000002e-05 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:44 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'26, mlcod=17'23)", 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.308042+0000", 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.967914902, 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010149810000000001, 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308042+0000", 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308042+0000", 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308042+0000", 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7400000000000003e-07 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308106+0000", 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3814999999999998e-05 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308116+0000", 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0210000000000001e-05 2024-09-12T07:19:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308121+0000", 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308158+0000", 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7011000000000002e-05 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308172+0000", 2024-09-12T07:19:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3915e-05 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.309014+0000", 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00084201299999999996 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.309057+0000", 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2460999999999999e-05 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'29)", 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.312716+0000", 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.963240342999999, 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00083828699999999995, 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.312716+0000", 2024-09-12T07:19:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.312716+0000", 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.312717+0000", 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6700000000000003e-07 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.312774+0000", 2024-09-12T07:19:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6981999999999998e-05 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.312783+0000", 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5489999999999995e-06 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.312788+0000", 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.34e-06 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.312816+0000", 2024-09-12T07:19:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8541999999999999e-05 2024-09-12T07:19:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.312832+0000", 2024-09-12T07:19:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6316000000000001e-05 2024-09-12T07:19:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.313523+0000", 2024-09-12T07:19:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00069101999999999998 2024-09-12T07:19:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.313554+0000", 2024-09-12T07:19:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970999999999999e-05 2024-09-12T07:19:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:64 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'42, mlcod=17'40)", 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.525261+0000", 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.750695802999999, 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00091668200000000004, 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525261+0000", 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525261+0000", 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525261+0000", 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8899999999999999e-07 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525266+0000", 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5530000000000003e-06 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525275+0000", 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525280+0000", 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.532e-06 2024-09-12T07:19:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525331+0000", 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1218000000000003e-05 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525352+0000", 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0605e-05 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526148+0000", 2024-09-12T07:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079593800000000005 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526177+0000", 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9662000000000002e-05 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.525781+0000", 2024-09-12T07:19:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.750175066000001, 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012042470000000001, 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525781+0000", 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525781+0000", 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525782+0000", 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0999999999999998e-07 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525786+0000", 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2250000000000002e-06 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525796+0000", 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9790000000000005e-06 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525801+0000", 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7149999999999997e-06 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525857+0000", 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5566000000000003e-05 2024-09-12T07:19:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525877+0000", 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0825000000000001e-05 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526963+0000", 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010860869999999999 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526986+0000", 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2240000000000001e-05 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.526323+0000", 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.749633615, 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010357579999999999, 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526323+0000", 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526323+0000", 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526323+0000", 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4900000000000001e-07 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526384+0000", 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0543e-05 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526392+0000", 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8800000000000008e-06 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526395+0000", 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6689999999999999e-06 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526437+0000", 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1999999999999998e-05 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526461+0000", 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3642000000000002e-05 2024-09-12T07:19:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.527335+0000", 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087443999999999998 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.527359+0000", 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3135000000000001e-05 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:72 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'50, mlcod=17'47)", 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.531002+0000", 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.744954321000002, 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011292909999999999, 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531002+0000", 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531002+0000", 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531003+0000", 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4499999999999997e-07 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531071+0000", 2024-09-12T07:19:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7937999999999999e-05 2024-09-12T07:19:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531080+0000", 2024-09-12T07:19:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1179999999999998e-06 2024-09-12T07:19:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531084+0000", 2024-09-12T07:19:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7299999999999996e-06 2024-09-12T07:19:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531147+0000", 2024-09-12T07:19:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2298000000000006e-05 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531165+0000", 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8468e-05 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.532104+0000", 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00093910599999999999 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.532131+0000", 2024-09-12T07:19:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7087999999999999e-05 2024-09-12T07:19:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:77 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'55, mlcod=17'53)", 2024-09-12T07:19:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.535644+0000", 2024-09-12T07:19:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.740312187000001, 2024-09-12T07:19:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00093193699999999998, 2024-09-12T07:19:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535644+0000", 2024-09-12T07:19:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535644+0000", 2024-09-12T07:19:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535645+0000", 2024-09-12T07:19:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5300000000000004e-07 2024-09-12T07:19:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535693+0000", 2024-09-12T07:19:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8279000000000003e-05 2024-09-12T07:19:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535703+0000", 2024-09-12T07:19:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0249e-05 2024-09-12T07:19:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535708+0000", 2024-09-12T07:19:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3370000000000001e-06 2024-09-12T07:19:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535750+0000", 2024-09-12T07:19:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1802999999999999e-05 2024-09-12T07:19:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535770+0000", 2024-09-12T07:19:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9890000000000001e-05 2024-09-12T07:19:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536550+0000", 2024-09-12T07:19:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00078073599999999997 2024-09-12T07:19:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536576+0000", 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.599e-05 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:78 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'56, mlcod=17'53)", 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.536150+0000", 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.739806766000001, 2024-09-12T07:19:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00085383599999999996, 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536150+0000", 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536150+0000", 2024-09-12T07:19:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536150+0000", 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4499999999999997e-07 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536221+0000", 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1079000000000002e-05 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536229+0000", 2024-09-12T07:19:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7430000000000002e-06 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536232+0000", 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3979999999999999e-06 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536272+0000", 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9968000000000001e-05 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536294+0000", 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1874000000000001e-05 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536983+0000", 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00068858899999999998 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.537004+0000", 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0740000000000001e-05 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.365 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.366 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:79 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-12T07:19:44.366 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.536630+0000", 2024-09-12T07:19:44.366 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.739326695999999, 2024-09-12T07:19:44.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00090458900000000002, 2024-09-12T07:19:44.366 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.366 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.366 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536630+0000", 2024-09-12T07:19:44.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536630+0000", 2024-09-12T07:19:44.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536630+0000", 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3799999999999997e-07 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536720+0000", 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9252999999999994e-05 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536730+0000", 2024-09-12T07:19:44.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536735+0000", 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6609999999999999e-06 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536775+0000", 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0450000000000001e-05 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536788+0000", 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2536e-05 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.537513+0000", 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072574200000000003 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.537534+0000", 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1098999999999999e-05 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.370 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.810835+0000", 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.465121450000002, 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095822099999999999, 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810835+0000", 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810835+0000", 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810835+0000", 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810880+0000", 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4286999999999998e-05 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810888+0000", 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3189999999999995e-06 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810892+0000", 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5810000000000001e-06 2024-09-12T07:19:44.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810927+0000", 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5771e-05 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810960+0000", 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2960999999999998e-05 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.811766+0000", 2024-09-12T07:19:44.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00080538400000000003 2024-09-12T07:19:44.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.811793+0000", 2024-09-12T07:19:44.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.747e-05 2024-09-12T07:19:44.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.375 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.375 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.375 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:44.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:19:44.377 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:44.378+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is set 2024-09-12T07:19:44.477 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:44.478+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:44.493 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:44.493 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:44.493 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:19:44.493 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:44.502 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:19:44.578 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:44.580+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:44.650 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:44.650 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:44.650 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:19:44.650 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:19:44.650 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:44.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:19:44.702 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:44.704+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:44.766 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:19:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:19:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:19:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4244.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-12T07:19:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:31.221243+0000", 2024-09-12T07:19:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.546933836999999, 2024-09-12T07:19:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00097513999999999999, 2024-09-12T07:19:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221243+0000", 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221243+0000", 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221243+0000", 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5800000000000001e-07 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221248+0000", 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0209999999999999e-06 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221264+0000", 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6180000000000001e-05 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221274+0000", 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9159999999999996e-06 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221346+0000", 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2021999999999996e-05 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221376+0000", 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9215999999999999e-05 2024-09-12T07:19:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.222162+0000", 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00078618199999999996 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.222218+0000", 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6444999999999999e-05 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4244.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-12T07:19:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:32.225799+0000", 2024-09-12T07:19:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.542377992, 2024-09-12T07:19:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010253199999999999, 2024-09-12T07:19:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225799+0000", 2024-09-12T07:19:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225799+0000", 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225799+0000", 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.05e-07 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225807+0000", 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.695e-06 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225826+0000", 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8657000000000001e-05 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225836+0000", 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0910999999999999e-05 2024-09-12T07:19:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225917+0000", 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0636000000000005e-05 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225936+0000", 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8873e-05 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.226764+0000", 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000828078 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.226824+0000", 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0265e-05 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-12T07:19:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:43.446759+0000", 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3214179939999999, 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049415209999999999, 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446759+0000", 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446759+0000", 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446761+0000", 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3769999999999999e-06 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446773+0000", 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2092e-05 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446795+0000", 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1923999999999999e-05 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446809+0000", 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3373999999999999e-05 2024-09-12T07:19:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446885+0000", 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5791999999999995e-05 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446892+0000", 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1080000000000004e-06 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.450365+0000", 2024-09-12T07:19:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0034735270000000001 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.450445+0000", 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.017e-05 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.450539+0000", 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4109000000000003e-05 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451187+0000", 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00064770500000000005 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451638+0000", 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00045091199999999998 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451672+0000", 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3695000000000002e-05 2024-09-12T07:19:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451700+0000", 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8736000000000001e-05 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:7 3.1a e25/24 ondisk, result = 0)", 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:43.451557+0000", 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.316620042, 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000152597, 2024-09-12T07:19:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451557+0000", 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451557+0000", 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451558+0000", 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2399999999999996e-07 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451562+0000", 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4689999999999999e-06 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451574+0000", 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1497e-05 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451582+0000", 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1049999999999995e-06 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451628+0000", 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6269e-05 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451635+0000", 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4899999999999997e-06 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451709+0000", 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4843000000000002e-05 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.456994+0000", 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.31118320300000002, 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003270443, 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.456994+0000", 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.456994+0000", 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.456996+0000", 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1169999999999998e-06 2024-09-12T07:19:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.457005+0000", 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4110000000000002e-06 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.457021+0000", 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6093e-05 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.457030+0000", 2024-09-12T07:19:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5350000000000004e-06 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.457094+0000", 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4309999999999999e-05 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.457101+0000", 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.458702+0000", 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016009749999999999 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.458773+0000", 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1086000000000003e-05 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.458867+0000", 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4642999999999995e-05 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.459514+0000", 2024-09-12T07:19:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00064651900000000002 2024-09-12T07:19:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460207+0000", 2024-09-12T07:19:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00069281099999999999 2024-09-12T07:19:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460238+0000", 2024-09-12T07:19:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1644999999999999e-05 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460264+0000", 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5755e-05 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:9 3.1a e27/24 ondisk, result = 0)", 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.460137+0000", 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.30803978399999998, 2024-09-12T07:19:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131736, 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460137+0000", 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460137+0000", 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460138+0000", 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4900000000000005e-07 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460142+0000", 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1919999999999998e-06 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460151+0000", 2024-09-12T07:19:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3670000000000003e-06 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460161+0000", 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3419999999999997e-06 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460197+0000", 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5948000000000002e-05 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460203+0000", 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3820000000000001e-06 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460269+0000", 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5755999999999999e-05 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:13 3.19 3:9a926e65:::rbd_object_map.1160e8dffb2f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.470201+0000", 2024-09-12T07:19:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.29797554599999998, 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001456855, 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 13 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470201+0000", 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470201+0000", 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470203+0000", 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.826e-06 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470213+0000", 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0088000000000001e-05 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470230+0000", 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6612e-05 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470237+0000", 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470290+0000", 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2879e-05 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470373+0000", 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2507000000000001e-05 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470491+0000", 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011830200000000001 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471166+0000", 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00067496899999999998 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471579+0000", 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000413316 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471628+0000", 2024-09-12T07:19:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8449000000000003e-05 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471658+0000", 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0707999999999998e-05 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:13 3.19 e26/24 ondisk, result = 0)", 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.471502+0000", 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.29667471200000001, 2024-09-12T07:19:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016567099999999999, 2024-09-12T07:19:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471502+0000", 2024-09-12T07:19:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471502+0000", 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471503+0000", 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1500000000000003e-07 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471507+0000", 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6239999999999999e-06 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471516+0000", 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5200000000000003e-06 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471523+0000", 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471564+0000", 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1726e-05 2024-09-12T07:19:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471574+0000", 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4530000000000008e-06 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471668+0000", 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4159000000000004e-05 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:23 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.477676+0000", 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.29050107800000002, 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000207767, 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477676+0000", 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477676+0000", 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477678+0000", 2024-09-12T07:19:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.871e-06 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477683+0000", 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5230000000000003e-06 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477695+0000", 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2106999999999999e-05 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477702+0000", 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1530000000000002e-06 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477758+0000", 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6758000000000001e-05 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477813+0000", 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4407000000000003e-05 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477884+0000", 2024-09-12T07:19:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0947999999999999e-05 2024-09-12T07:19:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:44.813 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:44.815 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:19:44.815 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:44.816+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:44.915 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:44.917+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:44.962 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:44.962 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:44.963 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:19:44.963 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:44.972 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:19:45.016 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:45.017+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:45.079 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:45.079 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:45.079 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:19:45.080 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:19:45.080 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:45.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:19:45.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:19:45.117 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:45.118+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:45.207 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:19:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:19:45.218 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:45.219+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:45.225 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:19:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:19:45.240 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:19:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:19:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:19:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4244.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-12T07:19:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:30.217471+0000", 2024-09-12T07:19:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.024099208000001, 2024-09-12T07:19:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014488159999999999, 2024-09-12T07:19:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217471+0000", 2024-09-12T07:19:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217471+0000", 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217472+0000", 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4200000000000002e-07 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217482+0000", 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0067e-05 2024-09-12T07:19:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217494+0000", 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2437e-05 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217522+0000", 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7929999999999999e-05 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.218022+0000", 2024-09-12T07:19:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00049994099999999995 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.218103+0000", 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1038999999999994e-05 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.218872+0000", 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00076892799999999999 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.218920+0000", 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8131999999999999e-05 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:12 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.467367+0000", 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.77420355399999996, 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027275540000000001, 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467367+0000", 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467367+0000", 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467368+0000", 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1519999999999999e-06 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467372+0000", 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1980000000000003e-06 2024-09-12T07:19:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467380+0000", 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.515e-06 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467387+0000", 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0570000000000004e-06 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467437+0000", 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9738999999999999e-05 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467447+0000", 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0227999999999999e-05 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.468920+0000", 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014732059999999999 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469023+0000", 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103087 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469168+0000", 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014458099999999999 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469640+0000", 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047246499999999998 2024-09-12T07:19:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470041+0000", 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040072399999999998 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470072+0000", 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1080000000000001e-05 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470095+0000", 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2521999999999999e-05 2024-09-12T07:19:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:12 3.9 e27/24 ondisk, result = 0)", 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.469978+0000", 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.77159227699999999, 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012064, 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469978+0000", 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469978+0000", 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469979+0000", 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8999999999999995e-07 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469983+0000", 2024-09-12T07:19:45.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5109999999999999e-06 2024-09-12T07:19:45.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469991+0000", 2024-09-12T07:19:45.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2640000000000008e-06 2024-09-12T07:19:45.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469996+0000", 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470034+0000", 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7780000000000001e-05 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470038+0000", 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3379999999999998e-06 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470099+0000", 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0761000000000003e-05 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:16 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.475283+0000", 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.76628720100000003, 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030047100000000002, 2024-09-12T07:19:45.256 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 16 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475283+0000", 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475283+0000", 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475285+0000", 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.66e-06 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475296+0000", 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0728e-05 2024-09-12T07:19:45.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475310+0000", 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4450000000000001e-05 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475317+0000", 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3640000000000004e-06 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475378+0000", 2024-09-12T07:19:45.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1428999999999997e-05 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475437+0000", 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8482e-05 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475584+0000", 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014735799999999999 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:17 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.475660+0000", 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.76591099600000001, 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000157804, 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 17 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475660+0000", 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475660+0000", 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475660+0000", 2024-09-12T07:19:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5899999999999995e-07 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475664+0000", 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3900000000000002e-06 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475672+0000", 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.405e-06 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475679+0000", 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4250000000000003e-06 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475718+0000", 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9419999999999999e-05 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475755+0000", 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7345999999999998e-05 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475817+0000", 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2058999999999997e-05 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:18 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.475883+0000", 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.76568759399999997, 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188918, 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 18 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475883+0000", 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475883+0000", 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475884+0000", 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3099999999999997e-07 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475890+0000", 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.314e-06 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475898+0000", 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8839999999999994e-06 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475904+0000", 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4280000000000001e-06 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475941+0000", 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6337999999999997e-05 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.475992+0000", 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1013000000000001e-05 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476072+0000", 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0210000000000006e-05 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:19 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.476146+0000", 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.76542471599999995, 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016202499999999999, 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476146+0000", 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476146+0000", 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476147+0000", 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8700000000000005e-07 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476156+0000", 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0410000000000003e-06 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476164+0000", 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1219999999999995e-06 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476169+0000", 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6980000000000003e-06 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476206+0000", 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6819000000000001e-05 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476242+0000", 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5982000000000002e-05 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476308+0000", 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5675999999999999e-05 2024-09-12T07:19:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:20 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.476665+0000", 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.76490556099999996, 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032382299999999999, 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476665+0000", 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476665+0000", 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476666+0000", 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0400000000000005e-07 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476670+0000", 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8940000000000003e-06 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476679+0000", 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1479999999999995e-06 2024-09-12T07:19:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476684+0000", 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9690000000000002e-06 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476721+0000", 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7571000000000003e-05 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476765+0000", 2024-09-12T07:19:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3534e-05 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.476989+0000", 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022390299999999999 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:21 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.477071+0000", 2024-09-12T07:19:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.76449980500000003, 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000256603, 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 21 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477071+0000", 2024-09-12T07:19:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477071+0000", 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477072+0000", 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5000000000000002e-07 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477076+0000", 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1300000000000003e-06 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477084+0000", 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1559999999999995e-06 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477088+0000", 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3599999999999998e-06 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477129+0000", 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0318999999999998e-05 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477170+0000", 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1860000000000002e-05 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477327+0000", 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000157028 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:22 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.477418+0000", 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.76415246199999998, 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016419199999999999, 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477418+0000", 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477418+0000", 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477419+0000", 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1200000000000002e-07 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477423+0000", 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7079999999999999e-06 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477430+0000", 2024-09-12T07:19:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477434+0000", 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9770000000000002e-06 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477472+0000", 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7249999999999997e-05 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477508+0000", 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.684e-05 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.477582+0000", 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3954000000000004e-05 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:24 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.478076+0000", 2024-09-12T07:19:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.76349433300000003, 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027164000000000002, 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 24 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478076+0000", 2024-09-12T07:19:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478076+0000", 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478077+0000", 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1299999999999998e-07 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478082+0000", 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5159999999999996e-06 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478095+0000", 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3030000000000001e-05 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478110+0000", 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5648e-05 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478170+0000", 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9500000000000003e-05 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478228+0000", 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8230999999999999e-05 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478348+0000", 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119802 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:25 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [watch watch cookie 140539785655440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.478500+0000", 2024-09-12T07:19:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.76307111400000005, 2024-09-12T07:19:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014004029999999999, 2024-09-12T07:19:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 25 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478500+0000", 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478500+0000", 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478500+0000", 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4000000000000001e-07 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478504+0000", 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1250000000000003e-06 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478514+0000", 2024-09-12T07:19:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3200000000000006e-06 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478518+0000", 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478572+0000", 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3829999999999998e-05 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478658+0000", 2024-09-12T07:19:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5891999999999997e-05 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478828+0000", 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017019800000000001 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479385+0000", 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000556278 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479764+0000", 2024-09-12T07:19:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037902999999999998 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479821+0000", 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7574000000000001e-05 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479900+0000", 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8771999999999997e-05 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:25 3.9 e27/24 ondisk, result = 0)", 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.479687+0000", 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.76188403299999996, 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021908999999999999, 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479687+0000", 2024-09-12T07:19:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479687+0000", 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479687+0000", 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7800000000000001e-07 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479691+0000", 2024-09-12T07:19:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6119999999999999e-06 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479698+0000", 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1810000000000004e-06 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479703+0000", 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5009999999999998e-06 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479749+0000", 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.604e-05 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479758+0000", 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.149e-06 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479906+0000", 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014792900000000001 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:45.302 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:45.302 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:19:45.302 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:19:45.303 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:19:45.304 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:19:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:19:45.304 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:19:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:19:45.305 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:19:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:19:45.311 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:19:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:19:45.319 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:45.320+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:45.328 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:19:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:19:45.414 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:45.414 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:45.419 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:45.421+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:45.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:19:45.521 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:45.523+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:45.571 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:45.572 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:45.580 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:19:45.622 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:45.623+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:45.687 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:45.688 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:45.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:19:45.741 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:45.743+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:45.817 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:45.817 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:45.824 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:45.825+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:45.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:19:45.925 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:45.926+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:45.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:19:46.026 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:46.027+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:46.091 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:46.092 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:46.100 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:19:46.126 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:46.128+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout:skipping zero length punch hole 2024-09-12T07:19:46.181 INFO:teuthology.orchestra.run.smithi046.stdout:truncating to largest ever: 0x3222595 2024-09-12T07:19:46.184 INFO:teuthology.orchestra.run.smithi046.stdout:2 trunc from 0x0 to 0x3222595 2024-09-12T07:19:46.189 INFO:teuthology.orchestra.run.smithi046.stdout:3 read 0x2be2923 thru 0x2bec36f (0x9a4d bytes) 2024-09-12T07:19:46.207 INFO:teuthology.orchestra.run.smithi046.stdout:4 write 0xda95101 thru 0xda9af2d (0x5e2d bytes) 2024-09-12T07:19:46.208 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:46.208 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:46.210 INFO:teuthology.orchestra.run.smithi046.stdout:5 writesame 0xa62bdad thru 0xa633344 data_size 0x47(0x7598 bytes) 2024-09-12T07:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout:6 punch from 0x3eb1174 to 0x3eb2acc, (0x1958 bytes) 2024-09-12T07:19:46.214 INFO:teuthology.orchestra.run.smithi046.stdout:7 writesame 0xc20fa6e thru 0xc216f97 data_size 0x3(0x752a bytes) 2024-09-12T07:19:46.216 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:19:46.220 INFO:teuthology.orchestra.run.smithi046.stdout:8 write 0xbac662 thru 0xbb23f5 (0x5d94 bytes) 2024-09-12T07:19:46.223 INFO:teuthology.orchestra.run.smithi046.stdout:9 read 0x17381e9 thru 0x1746b45 (0xe95d bytes) 2024-09-12T07:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout:12 writesame 0xeb8cc82 thru 0xeb954f6 data_size 0x1(0x8875 bytes) 2024-09-12T07:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout:14 read 0xea3a696 thru 0xea4658c (0xbef7 bytes) 2024-09-12T07:19:46.230 INFO:teuthology.orchestra.run.smithi046.stdout:15 writesame 0xe185969 thru 0xe18d872 data_size 0xe(0x7f0a bytes) 2024-09-12T07:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.233 INFO:teuthology.orchestra.run.smithi046.stdout:truncating to largest ever: 0xd95e712 2024-09-12T07:19:46.235 INFO:teuthology.orchestra.run.smithi046.stdout:17 trunc from 0xeb954f7 to 0xd95e712 2024-09-12T07:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout:18 read 0x1810e86 thru 0x181742e (0x65a9 bytes) 2024-09-12T07:19:46.245 INFO:teuthology.orchestra.run.smithi046.stdout:19 punch from 0xd223f8f to 0xd231f1b, (0xdf8c bytes) 2024-09-12T07:19:46.260 INFO:teuthology.orchestra.run.smithi046.stdout:20 trunc from 0xd95e712 to 0x57d908 2024-09-12T07:19:46.260 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:46.262+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:46.279 INFO:teuthology.orchestra.run.smithi046.stdout:23 write 0x181f6f6 thru 0x1828388 (0x8c93 bytes) 2024-09-12T07:19:46.290 INFO:teuthology.orchestra.run.smithi046.stdout:24 writesame 0x31e5941 thru 0x31f49fe data_size 0x5(0xf0be bytes) 2024-09-12T07:19:46.290 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.290 INFO:teuthology.orchestra.run.smithi046.stdout:28 read 0x175cbc4 thru 0x17639ff (0x6e3c bytes) 2024-09-12T07:19:46.295 INFO:teuthology.orchestra.run.smithi046.stdout:29 trunc from 0x31f49ff to 0x7ab34b5 2024-09-12T07:19:46.300 INFO:teuthology.orchestra.run.smithi046.stdout:30 write 0x7914fe8 thru 0x7923ef6 (0xef0f bytes) 2024-09-12T07:19:46.303 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.304 INFO:teuthology.orchestra.run.smithi046.stdout:34 read 0x6e974f2 thru 0x6ea4d2f (0xd83e bytes) 2024-09-12T07:19:46.304 INFO:teuthology.orchestra.run.smithi046.stdout:35 punch from 0x4d34a13 to 0x4d3bb45, (0x7132 bytes) 2024-09-12T07:19:46.305 INFO:teuthology.orchestra.run.smithi046.stdout:36 trunc from 0x7ab34b5 to 0x6610738 2024-09-12T07:19:46.323 INFO:teuthology.orchestra.run.smithi046.stdout:37 writesame 0x976b2c3 thru 0x977962a data_size 0x13(0xe368 bytes) 2024-09-12T07:19:46.324 INFO:teuthology.orchestra.run.smithi046.stdout:38 read 0x1391232 thru 0x13914e3 (0x2b2 bytes) 2024-09-12T07:19:46.324 INFO:teuthology.orchestra.run.smithi046.stdout:40 punch from 0x1ffe4dc to 0x2003499, (0x4fbd bytes) 2024-09-12T07:19:46.324 INFO:teuthology.orchestra.run.smithi046.stdout:41 write 0x54061d0 thru 0x540ad58 (0x4b89 bytes) 2024-09-12T07:19:46.325 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:46.325 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:46.328 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:46.330+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:46.334 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:19:46.335 INFO:teuthology.orchestra.run.smithi046.stdout:43 write 0xcc48e63 thru 0xcc56a07 (0xdba5 bytes) 2024-09-12T07:19:46.338 INFO:teuthology.orchestra.run.smithi046.stdout:44 read 0x4801a45 thru 0x480f588 (0xdb44 bytes) 2024-09-12T07:19:46.338 INFO:teuthology.orchestra.run.smithi046.stdout:45 read 0xb3e1ef4 thru 0xb3edd84 (0xbe91 bytes) 2024-09-12T07:19:46.338 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.339 INFO:teuthology.orchestra.run.smithi046.stdout:48 read 0x6e9f8d8 thru 0x6ea692a (0x7053 bytes) 2024-09-12T07:19:46.339 INFO:teuthology.orchestra.run.smithi046.stdout:49 read 0x5e0af63 thru 0x5e141ba (0x9258 bytes) 2024-09-12T07:19:46.339 INFO:teuthology.orchestra.run.smithi046.stdout:54 read 0x10c5f29 thru 0x10d02f2 (0xa3ca bytes) 2024-09-12T07:19:46.339 INFO:teuthology.orchestra.run.smithi046.stdout:55 write 0xa2f66ae thru 0xa301077 (0xa9ca bytes) 2024-09-12T07:19:46.341 INFO:teuthology.orchestra.run.smithi046.stdout:57 write 0x4b71dbc thru 0x4b7e6f5 (0xc93a bytes) 2024-09-12T07:19:46.344 INFO:teuthology.orchestra.run.smithi046.stdout:59 write 0x2228651 thru 0x22372b3 (0xec63 bytes) 2024-09-12T07:19:46.347 INFO:teuthology.orchestra.run.smithi046.stdout:60 write 0x8236af6 thru 0x8240937 (0x9e42 bytes) 2024-09-12T07:19:46.351 INFO:teuthology.orchestra.run.smithi046.stdout:61 punch from 0x18413d3 to 0x184a605, (0x9232 bytes) 2024-09-12T07:19:46.351 INFO:teuthology.orchestra.run.smithi046.stdout:63 punch from 0x79125cf to 0x7915a51, (0x3482 bytes) 2024-09-12T07:19:46.351 INFO:teuthology.orchestra.run.smithi046.stdout:64 read 0x1991c8 thru 0x19d0a9 (0x3ee2 bytes) 2024-09-12T07:19:46.351 INFO:teuthology.orchestra.run.smithi046.stdout:66 write 0x39f0541 thru 0x39fbcb5 (0xb775 bytes) 2024-09-12T07:19:46.353 INFO:teuthology.orchestra.run.smithi046.stdout:67 read 0xc466a3d thru 0xc46dd4c (0x7310 bytes) 2024-09-12T07:19:46.353 INFO:teuthology.orchestra.run.smithi046.stdout:68 writesame 0x17eeff0 thru 0x17efadf data_size 0x14(0xaf0 bytes) 2024-09-12T07:19:46.356 INFO:teuthology.orchestra.run.smithi046.stdout:69 write 0x67b4d9a thru 0x67ba8ec (0x5b53 bytes) 2024-09-12T07:19:46.359 INFO:teuthology.orchestra.run.smithi046.stdout:70 write 0x8d2ccc3 thru 0x8d2d043 (0x381 bytes) 2024-09-12T07:19:46.361 INFO:teuthology.orchestra.run.smithi046.stdout:71 write 0x3826c7b thru 0x382d795 (0x6b1b bytes) 2024-09-12T07:19:46.363 INFO:teuthology.orchestra.run.smithi046.stdout:72 write 0xbd07fe1 thru 0xbd09d5f (0x1d7f bytes) 2024-09-12T07:19:46.365 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.366 INFO:teuthology.orchestra.run.smithi046.stdout:76 read 0x7505a2a thru 0x750f531 (0x9b08 bytes) 2024-09-12T07:19:46.366 INFO:teuthology.orchestra.run.smithi046.stdout:78 read 0xbe32fe4 thru 0xbe3ce64 (0x9e81 bytes) 2024-09-12T07:19:46.379 INFO:teuthology.orchestra.run.smithi046.stdout:79 trunc from 0xcc56a08 to 0x58e50c 2024-09-12T07:19:46.406 INFO:teuthology.orchestra.run.smithi046.stdout:80 write 0x5b4eabf thru 0x5b5c9ae (0xdef0 bytes) 2024-09-12T07:19:46.409 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.409 INFO:teuthology.orchestra.run.smithi046.stdout:83 write 0x4bdfc77 thru 0x4be07a6 (0xb30 bytes) 2024-09-12T07:19:46.412 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.412 INFO:teuthology.orchestra.run.smithi046.stdout:88 write 0x339e653 thru 0x33a6f1f (0x88cd bytes) 2024-09-12T07:19:46.418 INFO:teuthology.orchestra.run.smithi046.stdout:91 trunc from 0x5b5c9af to 0x2d6b019 2024-09-12T07:19:46.427 INFO:teuthology.orchestra.run.smithi046.stdout:92 read 0x49826f thru 0x49b1ee (0x2f80 bytes) 2024-09-12T07:19:46.429 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:46.431+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:46.445 INFO:teuthology.orchestra.run.smithi046.stdout:93 write 0xdedd3a1 thru 0xdee50f5 (0x7d55 bytes) 2024-09-12T07:19:46.448 INFO:teuthology.orchestra.run.smithi046.stdout:94 read 0xc96ceb thru 0xca0078 (0x938e bytes) 2024-09-12T07:19:46.448 INFO:teuthology.orchestra.run.smithi046.stdout:95 punch from 0xba1a9dd to 0xba1c262, (0x1885 bytes) 2024-09-12T07:19:46.448 INFO:teuthology.orchestra.run.smithi046.stdout:96 write 0x7b635ff thru 0x7b72a3d (0xf43f bytes) 2024-09-12T07:19:46.452 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.452 INFO:teuthology.orchestra.run.smithi046.stdout:98 punch from 0x2de49a7 to 0x2dec31f, (0x7978 bytes) 2024-09-12T07:19:46.452 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.452 INFO:teuthology.orchestra.run.smithi046.stdout:100 punch from 0xdce8a11 to 0xdce949b, (0xa8a bytes) 2024-09-12T07:19:46.452 INFO:teuthology.orchestra.run.smithi046.stdout:101 read 0x47f35b1 thru 0x480237c (0xedcc bytes) 2024-09-12T07:19:46.452 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.452 INFO:teuthology.orchestra.run.smithi046.stdout:103 read 0x4575d8c thru 0x457f787 (0x99fc bytes) 2024-09-12T07:19:46.452 INFO:teuthology.orchestra.run.smithi046.stdout:104 write 0x4c85f7a thru 0x4c867e8 (0x86f bytes) 2024-09-12T07:19:46.456 INFO:teuthology.orchestra.run.smithi046.stdout:105 punch from 0x8d2e640 to 0x8d2fbba, (0x157a bytes) 2024-09-12T07:19:46.456 INFO:teuthology.orchestra.run.smithi046.stdout:106 punch from 0x1421927 to 0x1423fb8, (0x2691 bytes) 2024-09-12T07:19:46.456 INFO:teuthology.orchestra.run.smithi046.stdout:107 write 0xbfb6b7f thru 0xbfc52c4 (0xe746 bytes) 2024-09-12T07:19:46.458 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:19:46.458 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:19:46.458 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-12T07:19:46.458 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-12T07:19:46.458 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-12T07:19:46.458 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-12T07:19:46.459 INFO:teuthology.orchestra.run.smithi046.stdout:108 write 0x6e606bc thru 0x6e686a2 (0x7fe7 bytes) 2024-09-12T07:19:46.462 INFO:teuthology.orchestra.run.smithi046.stdout:110 write 0xb1926bc thru 0xb199c74 (0x75b9 bytes) 2024-09-12T07:19:46.465 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:19:46.466 INFO:teuthology.orchestra.run.smithi046.stdout:111 write 0xdc340d9 thru 0xdc34bf9 (0xb21 bytes) 2024-09-12T07:19:46.468 INFO:teuthology.orchestra.run.smithi046.stdout:112 read 0x343c9d1 thru 0x3441c83 (0x52b3 bytes) 2024-09-12T07:19:46.468 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.468 INFO:teuthology.orchestra.run.smithi046.stdout:115 read 0xac9f158 thru 0xaca96d8 (0xa581 bytes) 2024-09-12T07:19:46.468 INFO:teuthology.orchestra.run.smithi046.stdout:116 read 0x88e650a thru 0x88f45f8 (0xe0ef bytes) 2024-09-12T07:19:46.475 INFO:teuthology.orchestra.run.smithi046.stdout:117 trunc from 0xdee50f6 to 0x69a79b2 2024-09-12T07:19:46.485 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:46.485 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:46.488 INFO:teuthology.orchestra.run.smithi046.stdout:118 write 0x40827af thru 0x40835a9 (0xdfb bytes) 2024-09-12T07:19:46.491 INFO:teuthology.orchestra.run.smithi046.stdout:120 read 0x5f38c43 thru 0x5f38d3e (0xfc bytes) 2024-09-12T07:19:46.494 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:19:46.503 INFO:teuthology.orchestra.run.smithi046.stdout:121 write 0xc5ea3af thru 0xc5f8b01 (0xe753 bytes) 2024-09-12T07:19:46.507 INFO:teuthology.orchestra.run.smithi046.stdout:123 read 0x5e11c85 thru 0x5e1a80c (0x8b88 bytes) 2024-09-12T07:19:46.507 INFO:teuthology.orchestra.run.smithi046.stdout:125 write 0x3105042 thru 0x310f141 (0xa100 bytes) 2024-09-12T07:19:46.510 INFO:teuthology.orchestra.run.smithi046.stdout:129 punch from 0x31e8cef to 0x31ee988, (0x5c99 bytes) 2024-09-12T07:19:46.510 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.510 INFO:teuthology.orchestra.run.smithi046.stdout:131 read 0x95e7ba2 thru 0x95f7398 (0xf7f7 bytes) 2024-09-12T07:19:46.510 INFO:teuthology.orchestra.run.smithi046.stdout:132 write 0xc3c420f thru 0xc3c9e20 (0x5c12 bytes) 2024-09-12T07:19:46.513 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.513 INFO:teuthology.orchestra.run.smithi046.stdout:134 write 0x1093d14 thru 0x109b8ed (0x7bda bytes) 2024-09-12T07:19:46.516 INFO:teuthology.orchestra.run.smithi046.stdout:135 write 0x46389 thru 0x4952d (0x31a5 bytes) 2024-09-12T07:19:46.520 INFO:teuthology.orchestra.run.smithi046.stdout:136 write 0xa111d74 thru 0xa119cfa (0x7f87 bytes) 2024-09-12T07:19:46.522 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.534 INFO:teuthology.orchestra.run.smithi046.stdout:139 trunc from 0xc5f8b02 to 0x27f5018 2024-09-12T07:19:46.535 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:46.537+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:46.562 INFO:teuthology.orchestra.run.smithi046.stdout:142 write 0x8972acb thru 0x8977fb7 (0x54ed bytes) 2024-09-12T07:19:46.565 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.565 INFO:teuthology.orchestra.run.smithi046.stdout:144 read 0x3009527 thru 0x300decd (0x49a7 bytes) 2024-09-12T07:19:46.565 INFO:teuthology.orchestra.run.smithi046.stdout:145 punch from 0x3791498 to 0x37956d3, (0x423b bytes) 2024-09-12T07:19:46.565 INFO:teuthology.orchestra.run.smithi046.stdout:146 read 0x794c8ac thru 0x794d033 (0x788 bytes) 2024-09-12T07:19:46.566 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.566 INFO:teuthology.orchestra.run.smithi046.stdout:149 writesame 0x3167054 thru 0x3172ef9 data_size 0x17(0xbea6 bytes) 2024-09-12T07:19:46.569 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.572 INFO:teuthology.orchestra.run.smithi046.stdout:151 trunc from 0x8977fb8 to 0xb5837c3 2024-09-12T07:19:46.579 INFO:teuthology.orchestra.run.smithi046.stdout:152 write 0x267f27d thru 0x2682c99 (0x3a1d bytes) 2024-09-12T07:19:46.582 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.582 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.582 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.582 INFO:teuthology.orchestra.run.smithi046.stdout:157 read 0x2a78460 thru 0x2a793d6 (0xf77 bytes) 2024-09-12T07:19:46.587 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:46.587 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-12T07:19:46.588 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:46.591 INFO:teuthology.orchestra.run.smithi046.stdout:159 trunc from 0xb5837c3 to 0x416313a 2024-09-12T07:19:46.601 INFO:teuthology.orchestra.run.smithi046.stdout:163 read 0x3afb494 thru 0x3afe9b6 (0x3523 bytes) 2024-09-12T07:19:46.602 INFO:teuthology.orchestra.run.smithi046.stdout:165 writesame 0x2124b8e thru 0x21282e5 data_size 0x2c(0x3758 bytes) 2024-09-12T07:19:46.617 INFO:teuthology.orchestra.run.smithi046.stdout:166 write 0x85077ab thru 0x8516341 (0xeb97 bytes) 2024-09-12T07:19:46.620 INFO:teuthology.orchestra.run.smithi046.stdout:167 read 0x1b32cd0 thru 0x1b3589f (0x2bd0 bytes) 2024-09-12T07:19:46.621 INFO:teuthology.orchestra.run.smithi046.stdout:168 write 0xd8af9b thru 0xd97f80 (0xcfe6 bytes) 2024-09-12T07:19:46.624 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:46.624 INFO:teuthology.orchestra.run.smithi046.stdout:170 read 0x6f5809b thru 0x6f580a9 (0xf bytes) 2024-09-12T07:19:46.624 INFO:teuthology.orchestra.run.smithi046.stdout:171 read 0x12b6d47 thru 0x12c697a (0xfc34 bytes) 2024-09-12T07:19:46.631 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:46.633+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:46.635 INFO:teuthology.orchestra.run.smithi046.stdout:173 write 0xc7600dc thru 0xc761cd7 (0x1bfc bytes) 2024-09-12T07:19:46.638 INFO:teuthology.orchestra.run.smithi046.stdout:174 writesame 0xa57b01 thru 0xa59b13 data_size 0x17(0x2013 bytes) 2024-09-12T07:19:46.641 INFO:teuthology.orchestra.run.smithi046.stdout:176 read 0x9495ee2 thru 0x94a1012 (0xb131 bytes) 2024-09-12T07:19:46.644 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:46.644 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:46.651 INFO:teuthology.orchestra.run.smithi046.stdout:177 trunc from 0xc761cd8 to 0x430f107 2024-09-12T07:19:46.653 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:19:46.662 INFO:teuthology.orchestra.run.smithi046.stdout:178 punch from 0x3951d40 to 0x395d4ab, (0xb76b bytes) 2024-09-12T07:19:46.662 INFO:teuthology.orchestra.run.smithi046.stdout:180 punch from 0x2ff27fa to 0x2fff62a, (0xce30 bytes) 2024-09-12T07:19:46.662 INFO:teuthology.orchestra.run.smithi046.stdout:181 read 0x2b954b9 thru 0x2b95ec0 (0xa08 bytes) 2024-09-12T07:19:46.662 INFO:teuthology.orchestra.run.smithi046.stdout:182 writesame 0x251a2b7 thru 0x2525889 data_size 0x1(0xb5d3 bytes) 2024-09-12T07:19:46.681 INFO:teuthology.orchestra.run.smithi046.stdout:184 clone 1 order 21 su 1048576 sc 15 2024-09-12T07:19:46.732 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:46.733+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:46.761 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:46.761 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:46.769 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:19:46.832 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:46.834+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:46.876 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:46.876 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:46.884 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:19:46.968 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:46.970+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:47.031 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:47.031 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:47.033 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:47.035+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:47.040 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:19:47.135 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:47.136+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:47.185 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:47.185 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:47.194 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:19:47.236 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:47.237+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:47.337 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:47.338+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:47.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:19:47.342 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:47.342 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:47.350 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:19:47.438 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:47.439+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:47.461 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:19:47.495 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:19:47.495 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:19:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:19:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:19:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:19:47.539 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:47.540+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:47.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:19:47.553 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:19:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:19:47.571 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:19:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:19:47.573 INFO:teuthology.orchestra.run.smithi046.stdout:185 read 0x35863e4 thru 0x3594fab (0xebc8 bytes) 2024-09-12T07:19:47.573 INFO:teuthology.orchestra.run.smithi046.stdout:187 read 0x315d7d0 thru 0x3167e50 (0xa681 bytes) 2024-09-12T07:19:47.574 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:47.589 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:19:47.589 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:19:47.589 INFO:teuthology.orchestra.run.smithi046.stdout:193 write 0x5b15b6d thru 0x5b24c05 (0xf099 bytes) 2024-09-12T07:19:47.593 INFO:teuthology.orchestra.run.smithi046.stdout:194 read 0x221df74 thru 0x2221951 (0x39de bytes) 2024-09-12T07:19:47.593 INFO:teuthology.orchestra.run.smithi046.stdout:195 punch from 0x1a487fc to 0x1a5467b, (0xbe7f bytes) 2024-09-12T07:19:47.607 INFO:teuthology.orchestra.run.smithi046.stdout:196 writesame 0xaf9b5a5 thru 0xafa00dd data_size 0x31(0x4b39 bytes) 2024-09-12T07:19:47.607 INFO:teuthology.orchestra.run.smithi046.stdout:197 read 0x15dd62a thru 0x15e002d (0x2a04 bytes) 2024-09-12T07:19:47.607 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:47.607 INFO:teuthology.orchestra.run.smithi046.stdout:199 writesame 0x11f19b2 thru 0x11fbde1 data_size 0x8e(0xa430 bytes) 2024-09-12T07:19:47.612 INFO:teuthology.orchestra.run.smithi046.stdout:200 punch from 0x25591bb to 0x255ede3, (0x5c28 bytes) 2024-09-12T07:19:47.612 INFO:teuthology.orchestra.run.smithi046.stdout:202 read 0x794aae1 thru 0x794eb55 (0x4075 bytes) 2024-09-12T07:19:47.614 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:19:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:19:47.621 INFO:teuthology.orchestra.run.smithi046.stdout:203 write 0xcda119a thru 0xcdaf7c9 (0xe630 bytes) 2024-09-12T07:19:47.632 INFO:teuthology.orchestra.run.smithi046.stdout:205 writesame 0xe205f4c thru 0xe208032 data_size 0x1(0x20e7 bytes) 2024-09-12T07:19:47.632 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:19:47.632 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:19:47.633 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:47.633 INFO:teuthology.orchestra.run.smithi046.stdout:210 punch from 0x6174d29 to 0x618250f, (0xd7e6 bytes) 2024-09-12T07:19:47.633 INFO:teuthology.orchestra.run.smithi046.stdout:211 read 0x6b73c88 thru 0x6b775f6 (0x396f bytes) 2024-09-12T07:19:47.633 INFO:teuthology.orchestra.run.smithi046.stdout:212 write 0x2967692 thru 0x296844a (0xdb9 bytes) 2024-09-12T07:19:47.635 INFO:teuthology.orchestra.run.smithi046.stdout:213 read 0x36af924 thru 0x36b1ba1 (0x227e bytes) 2024-09-12T07:19:47.635 INFO:teuthology.orchestra.run.smithi046.stdout:214 write 0x8243ca8 thru 0x8252855 (0xebae bytes) 2024-09-12T07:19:47.638 INFO:teuthology.orchestra.run.smithi046.stdout:215 write 0xc7e7acd thru 0xc7eea74 (0x6fa8 bytes) 2024-09-12T07:19:47.639 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:47.640+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:47.643 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:47.643 INFO:teuthology.orchestra.run.smithi046.stdout:217 write 0xac0559d thru 0xac08bff (0x3663 bytes) 2024-09-12T07:19:47.648 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:47.648 INFO:teuthology.orchestra.run.smithi046.stdout:219 read 0xc503f53 thru 0xc50d167 (0x9215 bytes) 2024-09-12T07:19:47.650 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:19:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:19:47.663 INFO:teuthology.orchestra.run.smithi046.stdout:220 trunc from 0xe208033 to 0x131c123 2024-09-12T07:19:47.677 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:19:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:19:47.678 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:47.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:19:47.711 INFO:teuthology.orchestra.run.smithi046.stdout:222 writesame 0xac60934 thru 0xac6a4ce data_size 0x1f(0x9b9b bytes) 2024-09-12T07:19:47.711 INFO:teuthology.orchestra.run.smithi046.stdout:223 read 0x48262ba thru 0x482d1b1 (0x6ef8 bytes) 2024-09-12T07:19:47.718 INFO:teuthology.orchestra.run.smithi046.stdout:225 trunc from 0xac6a4cf to 0x3b2de39 2024-09-12T07:19:47.725 INFO:teuthology.orchestra.run.smithi046.stdout:226 read 0x1777f52 thru 0x177d721 (0x57d0 bytes) 2024-09-12T07:19:47.725 INFO:teuthology.orchestra.run.smithi046.stdout:229 read 0x66e384 thru 0x670dbe (0x2a3b bytes) 2024-09-12T07:19:47.726 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:47.726 INFO:teuthology.orchestra.run.smithi046.stdout:233 write 0x394bc5d thru 0x395909e (0xd442 bytes) 2024-09-12T07:19:47.740 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:47.742+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:47.748 INFO:teuthology.orchestra.run.smithi046.stdout:234 writesame 0xe6ebb8e thru 0xe6f6ea2 data_size 0x35(0xb315 bytes) 2024-09-12T07:19:47.761 INFO:teuthology.orchestra.run.smithi046.stdout:235 trunc from 0xe6f6ea3 to 0x30352d0 2024-09-12T07:19:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:239 write 0x6f3b591 thru 0x6f40316 (0x4d86 bytes) 2024-09-12T07:19:47.782 INFO:teuthology.orchestra.run.smithi046.stdout:240 read 0x26e8914 thru 0x26ed6cf (0x4dbc bytes) 2024-09-12T07:19:47.782 INFO:teuthology.orchestra.run.smithi046.stdout:243 read 0x59e9ddd thru 0x59f0124 (0x6348 bytes) 2024-09-12T07:19:47.782 INFO:teuthology.orchestra.run.smithi046.stdout:244 punch from 0x2c1b422 to 0x2c251f0, (0x9dce bytes) 2024-09-12T07:19:47.787 INFO:teuthology.orchestra.run.smithi046.stdout:245 trunc from 0x6f40317 to 0x20d84c7 2024-09-12T07:19:47.805 INFO:teuthology.orchestra.run.smithi046.stdout:246 punch from 0x1c6ed0f to 0x1c7bd53, (0xd044 bytes) 2024-09-12T07:19:47.827 INFO:teuthology.orchestra.run.smithi046.stdout:247 write 0xec639cc thru 0xec6bf90 (0x85c5 bytes) 2024-09-12T07:19:47.831 INFO:teuthology.orchestra.run.smithi046.stdout:248 read 0x36fd173 thru 0x370b66c (0xe4fa bytes) 2024-09-12T07:19:47.831 INFO:teuthology.orchestra.run.smithi046.stdout:249 write 0xa0e9ae thru 0xa1919e (0xa7f1 bytes) 2024-09-12T07:19:47.832 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:8 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'3, mlcod=17'1)", 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.198872+0000", 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.634217322000001, 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00090540399999999995, 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.198872+0000", 2024-09-12T07:19:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.198872+0000", 2024-09-12T07:19:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.198874+0000", 2024-09-12T07:19:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.716e-06 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.198887+0000", 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.257e-05 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.198914+0000", 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7736e-05 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.198932+0000", 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7617000000000001e-05 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.199060+0000", 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012835900000000001 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.199099+0000", 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9048999999999998e-05 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.199740+0000", 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000641016 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.199778+0000", 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7341e-05 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-12T07:19:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.210761+0000", 2024-09-12T07:19:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.622328531000001, 2024-09-12T07:19:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011487330000000001, 2024-09-12T07:19:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210761+0000", 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210761+0000", 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210762+0000", 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6000000000000002e-07 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210863+0000", 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100614 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210876+0000", 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3526999999999999e-05 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210888+0000", 2024-09-12T07:19:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2140000000000001e-05 2024-09-12T07:19:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210936+0000", 2024-09-12T07:19:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7753000000000002e-05 2024-09-12T07:19:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.210959+0000", 2024-09-12T07:19:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3394e-05 2024-09-12T07:19:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211855+0000", 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00089539600000000002 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211910+0000", 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5049000000000001e-05 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.211373+0000", 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.621716314, 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001078702, 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211373+0000", 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211373+0000", 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211374+0000", 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2600000000000002e-07 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211474+0000", 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9932999999999993e-05 2024-09-12T07:19:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211485+0000", 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1029999999999999e-05 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211495+0000", 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5829999999999996e-06 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211549+0000", 2024-09-12T07:19:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4452e-05 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211567+0000", 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8476999999999999e-05 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212385+0000", 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081729700000000003 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212452+0000", 2024-09-12T07:19:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7204000000000002e-05 2024-09-12T07:19:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:22 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'12, mlcod=17'8)", 2024-09-12T07:19:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.211865+0000", 2024-09-12T07:19:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.621224087999998, 2024-09-12T07:19:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099281999999999994, 2024-09-12T07:19:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211865+0000", 2024-09-12T07:19:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211865+0000", 2024-09-12T07:19:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211866+0000", 2024-09-12T07:19:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0500000000000003e-07 2024-09-12T07:19:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211973+0000", 2024-09-12T07:19:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106456 2024-09-12T07:19:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211985+0000", 2024-09-12T07:19:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2286000000000001e-05 2024-09-12T07:19:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.211991+0000", 2024-09-12T07:19:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-12T07:19:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212032+0000", 2024-09-12T07:19:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0528000000000002e-05 2024-09-12T07:19:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212047+0000", 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4625e-05 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212824+0000", 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00077776699999999995 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212858+0000", 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3973000000000002e-05 2024-09-12T07:19:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:23 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'13, mlcod=17'8)", 2024-09-12T07:19:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.212427+0000", 2024-09-12T07:19:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.620662214999999, 2024-09-12T07:19:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000904243, 2024-09-12T07:19:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212427+0000", 2024-09-12T07:19:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212427+0000", 2024-09-12T07:19:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212428+0000", 2024-09-12T07:19:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7899999999999997e-07 2024-09-12T07:19:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212482+0000", 2024-09-12T07:19:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3919999999999999e-05 2024-09-12T07:19:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212494+0000", 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1778e-05 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212500+0000", 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6239999999999999e-06 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212544+0000", 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4239000000000001e-05 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.212566+0000", 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2359000000000001e-05 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.213296+0000", 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00073030100000000004 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.213332+0000", 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5142999999999999e-05 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:34 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'20, mlcod=17'18)", 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.301699+0000", 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.531390432999999, 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001000302, 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301699+0000", 2024-09-12T07:19:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301699+0000", 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301713+0000", 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3517e-05 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301767+0000", 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3902e-05 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301811+0000", 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4523e-05 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301829+0000", 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8029999999999998e-05 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.301993+0000", 2024-09-12T07:19:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016428600000000001 2024-09-12T07:19:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.302017+0000", 2024-09-12T07:19:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3436999999999999e-05 2024-09-12T07:19:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.302667+0000", 2024-09-12T07:19:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00064969599999999995 2024-09-12T07:19:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.302699+0000", 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2910999999999997e-05 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:41 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'23, mlcod=17'19)", 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.305735+0000", 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.527355074999999, 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010266030000000001, 2024-09-12T07:19:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305735+0000", 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305735+0000", 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305735+0000", 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0200000000000002e-07 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305814+0000", 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8709999999999997e-05 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305823+0000", 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1589999999999999e-06 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305827+0000", 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1899999999999997e-06 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305863+0000", 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5648000000000002e-05 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.305878+0000", 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5611e-05 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.306716+0000", 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00083814200000000005 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.306761+0000", 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4641000000000002e-05 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:44 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'26, mlcod=17'23)", 2024-09-12T07:19:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.308042+0000", 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.525047997000001, 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010149810000000001, 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308042+0000", 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308042+0000", 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308042+0000", 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7400000000000003e-07 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308106+0000", 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3814999999999998e-05 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308116+0000", 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0210000000000001e-05 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308121+0000", 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308158+0000", 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7011000000000002e-05 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.308172+0000", 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3915e-05 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.309014+0000", 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00084201299999999996 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.309057+0000", 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2460999999999999e-05 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:64 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'42, mlcod=17'40)", 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.525261+0000", 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.307828898, 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00091668200000000004, 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525261+0000", 2024-09-12T07:19:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525261+0000", 2024-09-12T07:19:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525261+0000", 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8899999999999999e-07 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525266+0000", 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5530000000000003e-06 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525275+0000", 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525280+0000", 2024-09-12T07:19:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.532e-06 2024-09-12T07:19:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525331+0000", 2024-09-12T07:19:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1218000000000003e-05 2024-09-12T07:19:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525352+0000", 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0605e-05 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526148+0000", 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079593800000000005 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526177+0000", 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9662000000000002e-05 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-12T07:19:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.525781+0000", 2024-09-12T07:19:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.307308161000002, 2024-09-12T07:19:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012042470000000001, 2024-09-12T07:19:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525781+0000", 2024-09-12T07:19:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525781+0000", 2024-09-12T07:19:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525782+0000", 2024-09-12T07:19:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0999999999999998e-07 2024-09-12T07:19:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525786+0000", 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2250000000000002e-06 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525796+0000", 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9790000000000005e-06 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525801+0000", 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7149999999999997e-06 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525857+0000", 2024-09-12T07:19:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5566000000000003e-05 2024-09-12T07:19:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.525877+0000", 2024-09-12T07:19:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0825000000000001e-05 2024-09-12T07:19:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526963+0000", 2024-09-12T07:19:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010860869999999999 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526986+0000", 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2240000000000001e-05 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.526323+0000", 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.306766710000002, 2024-09-12T07:19:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010357579999999999, 2024-09-12T07:19:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526323+0000", 2024-09-12T07:19:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526323+0000", 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526323+0000", 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4900000000000001e-07 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526384+0000", 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0543e-05 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526392+0000", 2024-09-12T07:19:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8800000000000008e-06 2024-09-12T07:19:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526395+0000", 2024-09-12T07:19:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6689999999999999e-06 2024-09-12T07:19:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526437+0000", 2024-09-12T07:19:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1999999999999998e-05 2024-09-12T07:19:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.526461+0000", 2024-09-12T07:19:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3642000000000002e-05 2024-09-12T07:19:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.527335+0000", 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087443999999999998 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.527359+0000", 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3135000000000001e-05 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:72 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'50, mlcod=17'47)", 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.531002+0000", 2024-09-12T07:19:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.302087415999999, 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011292909999999999, 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531002+0000", 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531002+0000", 2024-09-12T07:19:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531003+0000", 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4499999999999997e-07 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531071+0000", 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7937999999999999e-05 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531080+0000", 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1179999999999998e-06 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531084+0000", 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7299999999999996e-06 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531147+0000", 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2298000000000006e-05 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.531165+0000", 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8468e-05 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.532104+0000", 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00093910599999999999 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.532131+0000", 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7087999999999999e-05 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:77 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'55, mlcod=17'53)", 2024-09-12T07:19:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.535644+0000", 2024-09-12T07:19:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.297445281999998, 2024-09-12T07:19:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00093193699999999998, 2024-09-12T07:19:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535644+0000", 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535644+0000", 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535645+0000", 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5300000000000004e-07 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535693+0000", 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8279000000000003e-05 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535703+0000", 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0249e-05 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535708+0000", 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3370000000000001e-06 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535750+0000", 2024-09-12T07:19:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1802999999999999e-05 2024-09-12T07:19:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.535770+0000", 2024-09-12T07:19:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9890000000000001e-05 2024-09-12T07:19:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536550+0000", 2024-09-12T07:19:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00078073599999999997 2024-09-12T07:19:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536576+0000", 2024-09-12T07:19:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.599e-05 2024-09-12T07:19:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:79 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.536630+0000", 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.296459791, 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00090458900000000002, 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536630+0000", 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536630+0000", 2024-09-12T07:19:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536630+0000", 2024-09-12T07:19:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3799999999999997e-07 2024-09-12T07:19:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536720+0000", 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9252999999999994e-05 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536730+0000", 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536735+0000", 2024-09-12T07:19:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6609999999999999e-06 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536775+0000", 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0450000000000001e-05 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.536788+0000", 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2536e-05 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.537513+0000", 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00072574200000000003 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.537534+0000", 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1098999999999999e-05 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4181.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:25.810835+0000", 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.022254544999999, 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095822099999999999, 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810835+0000", 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810835+0000", 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810835+0000", 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810880+0000", 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4286999999999998e-05 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810888+0000", 2024-09-12T07:19:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3189999999999995e-06 2024-09-12T07:19:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810892+0000", 2024-09-12T07:19:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5810000000000001e-06 2024-09-12T07:19:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810927+0000", 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5771e-05 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.810960+0000", 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2960999999999998e-05 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.811766+0000", 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00080538400000000003 2024-09-12T07:19:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:25.811793+0000", 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.747e-05 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.464984+0000", 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.3681060120000001, 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001831154, 2024-09-12T07:19:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.464984+0000", 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.464984+0000", 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.464986+0000", 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9429999999999999e-06 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.464996+0000", 2024-09-12T07:19:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0654e-05 2024-09-12T07:19:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.465014+0000", 2024-09-12T07:19:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-12T07:19:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.465023+0000", 2024-09-12T07:19:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9049999999999992e-06 2024-09-12T07:19:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.465095+0000", 2024-09-12T07:19:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1089999999999999e-05 2024-09-12T07:19:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.465258+0000", 2024-09-12T07:19:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016370800000000001 2024-09-12T07:19:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.465487+0000", 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022903299999999999 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.466383+0000", 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00089607399999999996 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.466694+0000", 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00031106900000000003 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.466757+0000", 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2439000000000003e-05 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.466815+0000", 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7819000000000003e-05 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:574 3.1b 3:d8fa3ef0:::rbd_data.116011b0bca6.000000000000001a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 423570~3513] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.636621+0000", 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.19646893900000001, 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001344011, 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 574 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.636621+0000", 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.636621+0000", 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.636623+0000", 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.401e-06 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.636635+0000", 2024-09-12T07:19:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1908e-05 2024-09-12T07:19:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.636657+0000", 2024-09-12T07:19:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2285000000000001e-05 2024-09-12T07:19:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.636669+0000", 2024-09-12T07:19:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2068e-05 2024-09-12T07:19:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.636732+0000", 2024-09-12T07:19:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2651999999999996e-05 2024-09-12T07:19:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.636832+0000", 2024-09-12T07:19:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100246 2024-09-12T07:19:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.636974+0000", 2024-09-12T07:19:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014228900000000001 2024-09-12T07:19:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.637678+0000", 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00070307900000000001 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.637894+0000", 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021684800000000001 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.637926+0000", 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1921999999999997e-05 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.637965+0000", 2024-09-12T07:19:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8312999999999997e-05 2024-09-12T07:19:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:582 3.1b 3:d8fa3ef0:::rbd_data.116011b0bca6.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.668009+0000", 2024-09-12T07:19:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.16508098099999999, 2024-09-12T07:19:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053356829999999999, 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 582 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668009+0000", 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668009+0000", 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668012+0000", 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.078e-06 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668022+0000", 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0608000000000001e-05 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668047+0000", 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4960000000000002e-05 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668060+0000", 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.325e-05 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668138+0000", 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7210000000000001e-05 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668220+0000", 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2158999999999997e-05 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668325+0000", 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104702 2024-09-12T07:19:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.669077+0000", 2024-09-12T07:19:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00075245399999999995 2024-09-12T07:19:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.673278+0000", 2024-09-12T07:19:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.004201443 2024-09-12T07:19:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.673310+0000", 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1520000000000003e-05 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.673344+0000", 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4298999999999998e-05 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:609 3.3 3:c2df91d2:::rbd_data.116011b0bca6.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.685399+0000", 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.14769091400000001, 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0041942790000000004, 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 609 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685399+0000", 2024-09-12T07:19:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685399+0000", 2024-09-12T07:19:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685401+0000", 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0600000000000002e-06 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685408+0000", 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.746e-06 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685425+0000", 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7017999999999999e-05 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685433+0000", 2024-09-12T07:19:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9580000000000007e-06 2024-09-12T07:19:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685502+0000", 2024-09-12T07:19:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8583000000000004e-05 2024-09-12T07:19:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685665+0000", 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016336399999999999 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685864+0000", 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019875599999999999 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687040+0000", 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011758199999999999 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.689481+0000", 2024-09-12T07:19:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024409380000000001 2024-09-12T07:19:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.689543+0000", 2024-09-12T07:19:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2382e-05 2024-09-12T07:19:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.689593+0000", 2024-09-12T07:19:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9654000000000003e-05 2024-09-12T07:19:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:614 3.17 3:e86dcc31:::rbd_data.116011b0bca6.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.686203+0000", 2024-09-12T07:19:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.146886342, 2024-09-12T07:19:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014994660000000001, 2024-09-12T07:19:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 614 2024-09-12T07:19:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686203+0000", 2024-09-12T07:19:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686203+0000", 2024-09-12T07:19:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686204+0000", 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.176e-06 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686210+0000", 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5439999999999998e-06 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686222+0000", 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1824999999999999e-05 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686238+0000", 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6178e-05 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686304+0000", 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5819000000000001e-05 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686420+0000", 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116691 2024-09-12T07:19:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686614+0000", 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019334300000000001 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687577+0000", 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00096282699999999998 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687601+0000", 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4077999999999999e-05 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687657+0000", 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6708e-05 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687703+0000", 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5277e-05 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:47.939 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:47.940 INFO:teuthology.orchestra.run.smithi046.stdout:250 trunc from 0xec6bf91 to 0x594a8ce 2024-09-12T07:19:47.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:19:47.942 INFO:teuthology.orchestra.run.smithi046.stdout:252 write 0x5c8daf0 thru 0x5c9732e (0x983f bytes) 2024-09-12T07:19:47.942 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:47.942 INFO:teuthology.orchestra.run.smithi046.stdout:255 trunc from 0x5c9732f to 0x40bd60b 2024-09-12T07:19:47.943 INFO:teuthology.orchestra.run.smithi046.stdout:256 read 0x27e0f9a thru 0x27e16a0 (0x707 bytes) 2024-09-12T07:19:47.943 INFO:teuthology.orchestra.run.smithi046.stdout:257 trunc from 0x40bd60b to 0x86c91fd 2024-09-12T07:19:47.943 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:47.943 INFO:teuthology.orchestra.run.smithi046.stdout:261 punch from 0x37e59c5 to 0x37e59e0, (0x1b bytes) 2024-09-12T07:19:47.943 INFO:teuthology.orchestra.run.smithi046.stdout:262 writesame 0xe414ec0 thru 0xe417ce3 data_size 0x2(0x2e24 bytes) 2024-09-12T07:19:47.943 INFO:teuthology.orchestra.run.smithi046.stdout:263 write 0x650b08d thru 0x651989a (0xe80e bytes) 2024-09-12T07:19:47.943 INFO:teuthology.orchestra.run.smithi046.stdout:264 trunc from 0xe417ce4 to 0x3cb01b8 2024-09-12T07:19:47.943 INFO:teuthology.orchestra.run.smithi046.stdout:266 read 0x1e4af2d thru 0x1e4ff09 (0x4fdd bytes) 2024-09-12T07:19:47.943 INFO:teuthology.orchestra.run.smithi046.stdout:267 read 0x2f67414 thru 0x2f6ec61 (0x784e bytes) 2024-09-12T07:19:47.943 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:47.943 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:47.942+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:47.944 INFO:teuthology.orchestra.run.smithi046.stdout:270 write 0x51591d5 thru 0x51651f6 (0xc022 bytes) 2024-09-12T07:19:47.946 INFO:teuthology.orchestra.run.smithi046.stdout:271 punch from 0x12bab7 to 0x135d81, (0xa2ca bytes) 2024-09-12T07:19:47.946 INFO:teuthology.orchestra.run.smithi046.stdout:273 read 0x454de9c thru 0x455105e (0x31c3 bytes) 2024-09-12T07:19:47.946 INFO:teuthology.orchestra.run.smithi046.stdout:274 read 0x137c8a6 thru 0x138b739 (0xee94 bytes) 2024-09-12T07:19:47.946 INFO:teuthology.orchestra.run.smithi046.stdout:275 read 0x30096fe thru 0x3010ccf (0x75d2 bytes) 2024-09-12T07:19:47.946 INFO:teuthology.orchestra.run.smithi046.stdout:276 read 0x4fd90ed thru 0x4fdd0f5 (0x4009 bytes) 2024-09-12T07:19:47.946 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:47.946 INFO:teuthology.orchestra.run.smithi046.stdout:278 punch from 0x1562105 to 0x1565751, (0x364c bytes) 2024-09-12T07:19:47.946 INFO:teuthology.orchestra.run.smithi046.stdout:279 writesame 0x2d994c2 thru 0x2d9e98b data_size 0x1(0x54ca bytes) 2024-09-12T07:19:47.958 INFO:teuthology.orchestra.run.smithi046.stdout:280 read 0x255d2b1 thru 0x2564f3e (0x7c8e bytes) 2024-09-12T07:19:47.958 INFO:teuthology.orchestra.run.smithi046.stdout:282 writesame 0xbf2fed thru 0xbf40e7 data_size 0x3(0x10fb bytes) 2024-09-12T07:19:47.959 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:47.959 INFO:teuthology.orchestra.run.smithi046.stdout:286 punch from 0x8c2b6e to 0x8cd685, (0xab17 bytes) 2024-09-12T07:19:47.965 INFO:teuthology.orchestra.run.smithi046.stdout:287 writesame 0x53f0933 thru 0x53fd6b3 data_size 0x1(0xcd81 bytes) 2024-09-12T07:19:47.965 INFO:teuthology.orchestra.run.smithi046.stdout:288 punch from 0x40cf4e0 to 0x40d1101, (0x1c21 bytes) 2024-09-12T07:19:47.967 INFO:teuthology.orchestra.run.smithi046.stdout:289 trunc from 0x53fd6b4 to 0x41bdfcb 2024-09-12T07:19:47.977 INFO:teuthology.orchestra.run.smithi046.stdout:290 write 0x3831d63 thru 0x383356f (0x180d bytes) 2024-09-12T07:19:47.984 INFO:teuthology.orchestra.run.smithi046.stdout:291 trunc from 0x41bdfcb to 0x7bedf08 2024-09-12T07:19:47.991 INFO:teuthology.orchestra.run.smithi046.stdout:293 trunc from 0x7bedf08 to 0x97976e3 2024-09-12T07:19:47.996 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.000 INFO:teuthology.orchestra.run.smithi046.stdout:295 trunc from 0x97976e3 to 0xc7dba78 2024-09-12T07:19:48.006 INFO:teuthology.orchestra.run.smithi046.stdout:296 write 0x8239c83 thru 0x8246f54 (0xd2d2 bytes) 2024-09-12T07:19:48.011 INFO:teuthology.orchestra.run.smithi046.stdout:297 punch from 0xa5b2f9f to 0xa5b430e, (0x136f bytes) 2024-09-12T07:19:48.011 INFO:teuthology.orchestra.run.smithi046.stdout:298 punch from 0x1903559 to 0x19065b9, (0x3060 bytes) 2024-09-12T07:19:48.011 INFO:teuthology.orchestra.run.smithi046.stdout:299 write 0xc456a52 thru 0xc45da1c (0x6fcb bytes) 2024-09-12T07:19:48.026 INFO:teuthology.orchestra.run.smithi046.stdout:301 trunc from 0xc7dba78 to 0x778c42d 2024-09-12T07:19:48.032 INFO:teuthology.orchestra.run.smithi046.stdout:303 writesame 0x5f26594 thru 0x5f2dcdc data_size 0x57(0x7749 bytes) 2024-09-12T07:19:48.035 INFO:teuthology.orchestra.run.smithi046.stdout:304 punch from 0x20cae8b to 0x20dae53, (0xffc8 bytes) 2024-09-12T07:19:48.041 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:48.043+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:48.045 INFO:teuthology.orchestra.run.smithi046.stdout:305 writesame 0xbe7d05c thru 0xbe81e8d data_size 0x1(0x4e32 bytes) 2024-09-12T07:19:48.045 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.045 INFO:teuthology.orchestra.run.smithi046.stdout:307 punch from 0xa9197de to 0xa919d5c, (0x57e bytes) 2024-09-12T07:19:48.045 INFO:teuthology.orchestra.run.smithi046.stdout:308 writesame 0x9f556ab thru 0x9f60bf3 data_size 0x1(0xb549 bytes) 2024-09-12T07:19:48.060 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:19:48.060 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:48.069 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:19:48.070 INFO:teuthology.orchestra.run.smithi046.stdout:309 read 0x98c803c thru 0x98d526f (0xd234 bytes) 2024-09-12T07:19:48.071 INFO:teuthology.orchestra.run.smithi046.stdout:310 punch from 0x7aa6c0d to 0x7ab4885, (0xdc78 bytes) 2024-09-12T07:19:48.071 INFO:teuthology.orchestra.run.smithi046.stdout:311 read 0x20ebc72 thru 0x20f0992 (0x4d21 bytes) 2024-09-12T07:19:48.071 INFO:teuthology.orchestra.run.smithi046.stdout:312 write 0x13118e3 thru 0x131acba (0x93d8 bytes) 2024-09-12T07:19:48.076 INFO:teuthology.orchestra.run.smithi046.stdout:313 trunc from 0xbe81e8e to 0x7752f87 2024-09-12T07:19:48.091 INFO:teuthology.orchestra.run.smithi046.stdout:314 trunc from 0x7752f87 to 0xccd9017 2024-09-12T07:19:48.096 INFO:teuthology.orchestra.run.smithi046.stdout:315 read 0xa35d4bf thru 0xa360bb5 (0x36f7 bytes) 2024-09-12T07:19:48.097 INFO:teuthology.orchestra.run.smithi046.stdout:316 read 0x1635009 thru 0x164446a (0xf462 bytes) 2024-09-12T07:19:48.097 INFO:teuthology.orchestra.run.smithi046.stdout:317 write 0xadaaf0 thru 0xae4586 (0x9a97 bytes) 2024-09-12T07:19:48.098 INFO:teuthology.orchestra.run.smithi046.stdout:318 punch from 0xe6b20a to 0xe7aa6a, (0xf860 bytes) 2024-09-12T07:19:48.099 INFO:teuthology.orchestra.run.smithi046.stdout:320 writesame 0x7a52c63 thru 0x7a6146e data_size 0x2(0xe80c bytes) 2024-09-12T07:19:48.116 INFO:teuthology.orchestra.run.smithi046.stdout:322 write 0x70dc9bf thru 0x70ded6b (0x23ad bytes) 2024-09-12T07:19:48.119 INFO:teuthology.orchestra.run.smithi046.stdout:323 write 0x5e086e8 thru 0x5e15184 (0xca9d bytes) 2024-09-12T07:19:48.123 INFO:teuthology.orchestra.run.smithi046.stdout:325 read 0x50c4f53 thru 0x50ca96c (0x5a1a bytes) 2024-09-12T07:19:48.125 INFO:teuthology.orchestra.run.smithi046.stdout:328 trunc from 0xccd9017 to 0xabc2365 2024-09-12T07:19:48.139 INFO:teuthology.orchestra.run.smithi046.stdout:329 trunc from 0xabc2365 to 0x37b10b0 2024-09-12T07:19:48.142 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:48.143+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:48.156 INFO:teuthology.orchestra.run.smithi046.stdout:330 punch from 0x2ff9dbb to 0x2ffb4b7, (0x16fc bytes) 2024-09-12T07:19:48.156 INFO:teuthology.orchestra.run.smithi046.stdout:331 read 0x13c53aa thru 0x13d14a8 (0xc0ff bytes) 2024-09-12T07:19:48.170 INFO:teuthology.orchestra.run.smithi046.stdout:332 write 0xb29c07b thru 0xb2a61d3 (0xa159 bytes) 2024-09-12T07:19:48.173 INFO:teuthology.orchestra.run.smithi046.stdout:333 punch from 0x5afe579 to 0x5afe746, (0x1cd bytes) 2024-09-12T07:19:48.182 INFO:teuthology.orchestra.run.smithi046.stdout:336 writesame 0xde8b43d thru 0xde982fb data_size 0x1(0xcebf bytes) 2024-09-12T07:19:48.182 INFO:teuthology.orchestra.run.smithi046.stdout:337 write 0x5877bad thru 0x58840cb (0xc51f bytes) 2024-09-12T07:19:48.185 INFO:teuthology.orchestra.run.smithi046.stdout:338 write 0x792829 thru 0x79bce4 (0x94bc bytes) 2024-09-12T07:19:48.186 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.187 INFO:teuthology.orchestra.run.smithi046.stdout:340 writesame 0x676a156 thru 0x676cc73 data_size 0x1(0x2b1e bytes) 2024-09-12T07:19:48.189 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:19:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:19:48.189 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:48.194 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.194 INFO:teuthology.orchestra.run.smithi046.stdout:342 punch from 0xd1accd9 to 0xd1b162c, (0x4953 bytes) 2024-09-12T07:19:48.195 INFO:teuthology.orchestra.run.smithi046.stdout:343 write 0xcc23447 thru 0xcc2f528 (0xc0e2 bytes) 2024-09-12T07:19:48.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:19:48.198 INFO:teuthology.orchestra.run.smithi046.stdout:344 read 0x908cdfc thru 0x9097855 (0xaa5a bytes) 2024-09-12T07:19:48.198 INFO:teuthology.orchestra.run.smithi046.stdout:345 write 0x4db0fd3 thru 0x4dbc519 (0xb547 bytes) 2024-09-12T07:19:48.201 INFO:teuthology.orchestra.run.smithi046.stdout:346 read 0xf450f6 thru 0xf470ba (0x1fc5 bytes) 2024-09-12T07:19:48.201 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.201 INFO:teuthology.orchestra.run.smithi046.stdout:348 read 0x22b95ff thru 0x22c57a4 (0xc1a6 bytes) 2024-09-12T07:19:48.201 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.201 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.201 INFO:teuthology.orchestra.run.smithi046.stdout:352 writesame 0x54576b3 thru 0x5466bae data_size 0x2(0xf4fc bytes) 2024-09-12T07:19:48.220 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.220 INFO:teuthology.orchestra.run.smithi046.stdout:354 writesame 0x810b245 thru 0x811830b data_size 0x1d(0xd0c7 bytes) 2024-09-12T07:19:48.225 INFO:teuthology.orchestra.run.smithi046.stdout:355 punch from 0xb7c9199 to 0xb7d093b, (0x77a2 bytes) 2024-09-12T07:19:48.225 INFO:teuthology.orchestra.run.smithi046.stdout:356 writesame 0xa3be3cb thru 0xa3c23a7 data_size 0x1(0x3fdd bytes) 2024-09-12T07:19:48.233 INFO:teuthology.orchestra.run.smithi046.stdout:357 write 0x737db8 thru 0x7472fa (0xf543 bytes) 2024-09-12T07:19:48.235 INFO:teuthology.orchestra.run.smithi046.stdout:358 read 0x4d85e81 thru 0x4d8b774 (0x58f4 bytes) 2024-09-12T07:19:48.236 INFO:teuthology.orchestra.run.smithi046.stdout:360 read 0x66cde82 thru 0x66d03be (0x253d bytes) 2024-09-12T07:19:48.236 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.236 INFO:teuthology.orchestra.run.smithi046.stdout:363 punch from 0x1bbf75e to 0x1bc77e6, (0x8088 bytes) 2024-09-12T07:19:48.236 INFO:teuthology.orchestra.run.smithi046.stdout:364 write 0x6c0188c thru 0x6c0c7fd (0xaf72 bytes) 2024-09-12T07:19:48.240 INFO:teuthology.orchestra.run.smithi046.stdout:365 write 0x2be5453 thru 0x2bf4033 (0xebe1 bytes) 2024-09-12T07:19:48.242 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:48.244+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:48.245 INFO:teuthology.orchestra.run.smithi046.stdout:366 write 0x80b2253 thru 0x80b6f5c (0x4d0a bytes) 2024-09-12T07:19:48.248 INFO:teuthology.orchestra.run.smithi046.stdout:367 write 0x9683922 thru 0x9692da4 (0xf483 bytes) 2024-09-12T07:19:48.254 INFO:teuthology.orchestra.run.smithi046.stdout:369 read 0x5d1a497 thru 0x5d1cebf (0x2a29 bytes) 2024-09-12T07:19:48.261 INFO:teuthology.orchestra.run.smithi046.stdout:370 writesame 0xee4cbca thru 0xee5b240 data_size 0x1(0xe677 bytes) 2024-09-12T07:19:48.261 INFO:teuthology.orchestra.run.smithi046.stdout:371 punch from 0xa7a9e00 to 0xa7ad753, (0x3953 bytes) 2024-09-12T07:19:48.261 INFO:teuthology.orchestra.run.smithi046.stdout:372 read 0x6f8af9a thru 0x6f92574 (0x75db bytes) 2024-09-12T07:19:48.262 INFO:teuthology.orchestra.run.smithi046.stdout:373 write 0x4f0bdd5 thru 0x4f19295 (0xd4c1 bytes) 2024-09-12T07:19:48.263 INFO:teuthology.orchestra.run.smithi046.stdout:375 writesame 0xad3a694 thru 0xad3ba9b data_size 0x4(0x1408 bytes) 2024-09-12T07:19:48.267 INFO:teuthology.orchestra.run.smithi046.stdout:376 write 0x45f75b8 thru 0x46066f7 (0xf140 bytes) 2024-09-12T07:19:48.270 INFO:teuthology.orchestra.run.smithi046.stdout:377 write 0xdf9370a thru 0xdfa2b88 (0xf47f bytes) 2024-09-12T07:19:48.273 INFO:teuthology.orchestra.run.smithi046.stdout:379 read 0x2893654 thru 0x28a136d (0xdd1a bytes) 2024-09-12T07:19:48.273 INFO:teuthology.orchestra.run.smithi046.stdout:380 read 0x680a577 thru 0x6817697 (0xd121 bytes) 2024-09-12T07:19:48.273 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.273 INFO:teuthology.orchestra.run.smithi046.stdout:386 writesame 0x92182ea thru 0x9227e6d data_size 0x2(0xfb84 bytes) 2024-09-12T07:19:48.290 INFO:teuthology.orchestra.run.smithi046.stdout:389 read 0xaf7b83c thru 0xaf8b21d (0xf9e2 bytes) 2024-09-12T07:19:48.290 INFO:teuthology.orchestra.run.smithi046.stdout:390 read 0xb288995 thru 0xb28b95b (0x2fc7 bytes) 2024-09-12T07:19:48.291 INFO:teuthology.orchestra.run.smithi046.stdout:391 writesame 0xa6b2dd3 thru 0xa6b6f98 data_size 0x1(0x41c6 bytes) 2024-09-12T07:19:48.300 INFO:teuthology.orchestra.run.smithi046.stdout:392 write 0xbaf420 thru 0xbb18ce (0x24af bytes) 2024-09-12T07:19:48.302 INFO:teuthology.orchestra.run.smithi046.stdout:393 read 0x68c5a14 thru 0x68c9cab (0x4298 bytes) 2024-09-12T07:19:48.302 INFO:teuthology.orchestra.run.smithi046.stdout:396 writesame 0xbf15e20 thru 0xbf1d7a4 data_size 0x1(0x7985 bytes) 2024-09-12T07:19:48.313 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:19:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:19:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:19:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4244.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-12T07:19:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:31.221243+0000", 2024-09-12T07:19:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.091968672, 2024-09-12T07:19:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00097513999999999999, 2024-09-12T07:19:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221243+0000", 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221243+0000", 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221243+0000", 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5800000000000001e-07 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221248+0000", 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0209999999999999e-06 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221264+0000", 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6180000000000001e-05 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221274+0000", 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9159999999999996e-06 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221346+0000", 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2021999999999996e-05 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.221376+0000", 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9215999999999999e-05 2024-09-12T07:19:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.222162+0000", 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00078618199999999996 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:31.222218+0000", 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6444999999999999e-05 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4244.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:32.225799+0000", 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 16.087412827000001, 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010253199999999999, 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225799+0000", 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225799+0000", 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225799+0000", 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.05e-07 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225807+0000", 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.695e-06 2024-09-12T07:19:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225826+0000", 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8657000000000001e-05 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225836+0000", 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0910999999999999e-05 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225917+0000", 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0636000000000005e-05 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.225936+0000", 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8873e-05 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.226764+0000", 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000828078 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:32.226824+0000", 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0265e-05 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:43.446759+0000", 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.866452829, 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049415209999999999, 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446759+0000", 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446759+0000", 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446761+0000", 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3769999999999999e-06 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446773+0000", 2024-09-12T07:19:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2092e-05 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446795+0000", 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1923999999999999e-05 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446809+0000", 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3373999999999999e-05 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446885+0000", 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5791999999999995e-05 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.446892+0000", 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1080000000000004e-06 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.450365+0000", 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0034735270000000001 2024-09-12T07:19:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.450445+0000", 2024-09-12T07:19:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.017e-05 2024-09-12T07:19:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.450539+0000", 2024-09-12T07:19:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4109000000000003e-05 2024-09-12T07:19:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451187+0000", 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00064770500000000005 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451638+0000", 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00045091199999999998 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451672+0000", 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3695000000000002e-05 2024-09-12T07:19:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:43.451700+0000", 2024-09-12T07:19:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8736000000000001e-05 2024-09-12T07:19:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.456994+0000", 2024-09-12T07:19:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.8562180379999997, 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003270443, 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.456994+0000", 2024-09-12T07:19:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.456994+0000", 2024-09-12T07:19:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.456996+0000", 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1169999999999998e-06 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.457005+0000", 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4110000000000002e-06 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.457021+0000", 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6093e-05 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.457030+0000", 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5350000000000004e-06 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.457094+0000", 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4309999999999999e-05 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.457101+0000", 2024-09-12T07:19:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.458702+0000", 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016009749999999999 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.458773+0000", 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1086000000000003e-05 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.458867+0000", 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4642999999999995e-05 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.459514+0000", 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00064651900000000002 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460207+0000", 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00069281099999999999 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460238+0000", 2024-09-12T07:19:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1644999999999999e-05 2024-09-12T07:19:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.460264+0000", 2024-09-12T07:19:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5755e-05 2024-09-12T07:19:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:13 3.19 3:9a926e65:::rbd_object_map.1160e8dffb2f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.470201+0000", 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.843010381, 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001456855, 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 13 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470201+0000", 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470201+0000", 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470203+0000", 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.826e-06 2024-09-12T07:19:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470213+0000", 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0088000000000001e-05 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470230+0000", 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6612e-05 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470237+0000", 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470290+0000", 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2879e-05 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470373+0000", 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2507000000000001e-05 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470491+0000", 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011830200000000001 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471166+0000", 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00067496899999999998 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471579+0000", 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000413316 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471628+0000", 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8449000000000003e-05 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.471658+0000", 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0707999999999998e-05 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:489 3.19 3:9a926e65:::rbd_object_map.1160e8dffb2f:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.553355+0000", 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.759856423, 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001173636, 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 489 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.553355+0000", 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.553355+0000", 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.553357+0000", 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.528e-06 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.553366+0000", 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6819999999999992e-06 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.553380+0000", 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4863e-05 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.553397+0000", 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6310000000000001e-05 2024-09-12T07:19:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.553455+0000", 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7926000000000001e-05 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.553525+0000", 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0210999999999996e-05 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.553643+0000", 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011857 2024-09-12T07:19:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.554227+0000", 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00058335199999999998 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.554456+0000", 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022964099999999999 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.554489+0000", 2024-09-12T07:19:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2453000000000003e-05 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.554529+0000", 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0099999999999999e-05 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:538 3.4 3:220e9991:::rbd_data.116011b0bca6.000000000000002e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 88941~61593] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.593851+0000", 2024-09-12T07:19:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.71936113800000001, 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001698766, 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 538 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.593851+0000", 2024-09-12T07:19:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.593851+0000", 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.593851+0000", 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9400000000000004e-07 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.593892+0000", 2024-09-12T07:19:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0639999999999997e-05 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.593903+0000", 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0764999999999999e-05 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.593910+0000", 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1330000000000001e-06 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.593969+0000", 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8989000000000001e-05 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.594080+0000", 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011112000000000001 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.594187+0000", 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107154 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.595138+0000", 2024-09-12T07:19:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000950565 2024-09-12T07:19:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.595482+0000", 2024-09-12T07:19:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034442899999999999 2024-09-12T07:19:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.595519+0000", 2024-09-12T07:19:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6325000000000002e-05 2024-09-12T07:19:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.595549+0000", 2024-09-12T07:19:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0852000000000002e-05 2024-09-12T07:19:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:583 3.4 3:220e9991:::rbd_data.116011b0bca6.000000000000002e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.668009+0000", 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64520309799999997, 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013996729999999999, 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 583 2024-09-12T07:19:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668009+0000", 2024-09-12T07:19:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668009+0000", 2024-09-12T07:19:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668013+0000", 2024-09-12T07:19:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1729999999999997e-06 2024-09-12T07:19:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668022+0000", 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8079999999999994e-06 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668044+0000", 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1916000000000002e-05 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668060+0000", 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6844e-05 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668131+0000", 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0467000000000001e-05 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668199+0000", 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7937999999999999e-05 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668278+0000", 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9268000000000001e-05 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668987+0000", 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00070848999999999999 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.669288+0000", 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000301031 2024-09-12T07:19:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.669351+0000", 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3278e-05 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.669408+0000", 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7460000000000002e-05 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:611 3.8 e28/24 3:10500085:::rbd_data.116011b0bca6.0000000000000004:head v 28'4, mlcod=27'2)", 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.686444+0000", 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.62676745, 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001046024, 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686444+0000", 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686444+0000", 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686446+0000", 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.654e-06 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686454+0000", 2024-09-12T07:19:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8010000000000004e-06 2024-09-12T07:19:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686475+0000", 2024-09-12T07:19:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1614e-05 2024-09-12T07:19:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686485+0000", 2024-09-12T07:19:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1360000000000003e-06 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686546+0000", 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1468000000000002e-05 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686567+0000", 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1030999999999999e-05 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687439+0000", 2024-09-12T07:19:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087222600000000003 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687490+0000", 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1094000000000002e-05 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:615 3.8 e28/24 3:14ff501c:::rbd_data.116011b0bca6.000000000000000a:head v 28'5, mlcod=27'2)", 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.687076+0000", 2024-09-12T07:19:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.62613577499999995, 2024-09-12T07:19:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012058800000000001, 2024-09-12T07:19:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687076+0000", 2024-09-12T07:19:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687076+0000", 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687077+0000", 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4300000000000002e-07 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687095+0000", 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7858000000000001e-05 2024-09-12T07:19:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687105+0000", 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0002e-05 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687111+0000", 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6089999999999997e-06 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687152+0000", 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0670000000000002e-05 2024-09-12T07:19:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687168+0000", 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6495999999999999e-05 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.688241+0000", 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010726399999999999 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.688282+0000", 2024-09-12T07:19:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0862000000000003e-05 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:618 3.4 3:20179eeb:::rbd_data.116011b0bca6.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.687101+0000", 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.62611101199999997, 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016541240000000001, 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 618 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687101+0000", 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687101+0000", 2024-09-12T07:19:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687102+0000", 2024-09-12T07:19:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.64e-06 2024-09-12T07:19:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687109+0000", 2024-09-12T07:19:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5390000000000004e-06 2024-09-12T07:19:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687121+0000", 2024-09-12T07:19:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2160999999999999e-05 2024-09-12T07:19:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687443+0000", 2024-09-12T07:19:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032209600000000001 2024-09-12T07:19:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687507+0000", 2024-09-12T07:19:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4029000000000006e-05 2024-09-12T07:19:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687644+0000", 2024-09-12T07:19:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000136651 2024-09-12T07:19:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:19:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687827+0000", 2024-09-12T07:19:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018305799999999999 2024-09-12T07:19:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.688516+0000", 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00068903300000000003 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.688689+0000", 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017338400000000001 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.688723+0000", 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3528000000000002e-05 2024-09-12T07:19:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.688755+0000", 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2005000000000003e-05 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:705 3.1a 3:5c8a85be:::rbd_data.116011b0bca6.0000000000000074:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1456588~34245] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.830864+0000", 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.482348261, 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002472676, 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 705 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.830864+0000", 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.830864+0000", 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.830864+0000", 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5099999999999999e-07 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.830878+0000", 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3379e-05 2024-09-12T07:19:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.830888+0000", 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0851e-05 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.830894+0000", 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7529999999999998e-06 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.830949+0000", 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4542000000000001e-05 2024-09-12T07:19:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.831041+0000", 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1908000000000002e-05 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.831145+0000", 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010396299999999999 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.831911+0000", 2024-09-12T07:19:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00076630899999999996 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.833261+0000", 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013502620000000001 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.833295+0000", 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.349e-05 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.833336+0000", 2024-09-12T07:19:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1467999999999997e-05 2024-09-12T07:19:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:706 3.8 e28/24 3:14ff501c:::rbd_data.116011b0bca6.000000000000000a:head v 28'6, mlcod=28'4)", 2024-09-12T07:19:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.834074+0000", 2024-09-12T07:19:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.47913777699999999, 2024-09-12T07:19:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00098275100000000002, 2024-09-12T07:19:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.834074+0000", 2024-09-12T07:19:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.834074+0000", 2024-09-12T07:19:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.834075+0000", 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4900000000000005e-07 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.834115+0000", 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0229999999999999e-05 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.834126+0000", 2024-09-12T07:19:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0806e-05 2024-09-12T07:19:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.834132+0000", 2024-09-12T07:19:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8669999999999999e-06 2024-09-12T07:19:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.834186+0000", 2024-09-12T07:19:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4228999999999999e-05 2024-09-12T07:19:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.834211+0000", 2024-09-12T07:19:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4890000000000001e-05 2024-09-12T07:19:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.835029+0000", 2024-09-12T07:19:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081788 2024-09-12T07:19:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.835057+0000", 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8099999999999999e-05 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:708 3.1a 3:5c8a85be:::rbd_data.116011b0bca6.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.847868+0000", 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.46534357199999998, 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025106059999999999, 2024-09-12T07:19:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 708 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.847868+0000", 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.847868+0000", 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.847871+0000", 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3719999999999999e-06 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.847881+0000", 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0172e-05 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.847902+0000", 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1478999999999999e-05 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.847919+0000", 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6786000000000002e-05 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.847999+0000", 2024-09-12T07:19:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9824e-05 2024-09-12T07:19:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.848067+0000", 2024-09-12T07:19:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8262000000000005e-05 2024-09-12T07:19:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.848148+0000", 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0759000000000004e-05 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.848681+0000", 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00053323200000000002 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.850315+0000", 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00163389 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.850349+0000", 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3760999999999997e-05 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.850379+0000", 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0069e-05 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:784 3.0 3:067c6c85:::rbd_data.116011b0bca6.0000000000000024:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1413589~49186] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.945856+0000", 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.36735537499999998, 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001662686, 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 784 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.945856+0000", 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.945856+0000", 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.945858+0000", 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8560000000000001e-06 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.945895+0000", 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7217e-05 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.945925+0000", 2024-09-12T07:19:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9356000000000002e-05 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.945979+0000", 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4032e-05 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.946035+0000", 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5788999999999997e-05 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.946134+0000", 2024-09-12T07:19:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9233000000000003e-05 2024-09-12T07:19:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:19:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.946232+0000", 2024-09-12T07:19:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7856000000000006e-05 2024-09-12T07:19:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.947041+0000", 2024-09-12T07:19:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00080969199999999999 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.947463+0000", 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00042158499999999998 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.947495+0000", 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1732e-05 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.947519+0000", 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4338e-05 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:799 3.0 3:067c6c85:::rbd_data.116011b0bca6.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.972169+0000", 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.34104289700000001, 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001213967, 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 799 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.972169+0000", 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.972169+0000", 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.972171+0000", 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.035e-06 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.972185+0000", 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3611999999999999e-05 2024-09-12T07:19:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.972206+0000", 2024-09-12T07:19:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1112e-05 2024-09-12T07:19:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.972215+0000", 2024-09-12T07:19:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3000000000000007e-06 2024-09-12T07:19:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.972285+0000", 2024-09-12T07:19:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9610000000000006e-05 2024-09-12T07:19:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.972363+0000", 2024-09-12T07:19:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8296999999999999e-05 2024-09-12T07:19:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.972437+0000", 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4302999999999997e-05 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.973218+0000", 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00078081899999999998 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.973312+0000", 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4291999999999998e-05 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.973347+0000", 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4909000000000001e-05 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.973383+0000", 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5678e-05 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:892 3.8 e28/24 3:14ff501c:::rbd_data.116011b0bca6.000000000000000a:head v 28'8, mlcod=28'6)", 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.100210+0000", 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.21300222099999999, 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00091056999999999998, 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.100210+0000", 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.100210+0000", 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.100210+0000", 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3699999999999999e-07 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.100247+0000", 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6287e-05 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.100257+0000", 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0552e-05 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.100262+0000", 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9570000000000001e-06 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.100319+0000", 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6563e-05 2024-09-12T07:19:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.100344+0000", 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5099000000000001e-05 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.101087+0000", 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00074324099999999995 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.101120+0000", 2024-09-12T07:19:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2934e-05 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:935 3.4 3:276183f2:::rbd_data.116011b0bca6.0000000000000058:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1687675~41305] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.174781+0000", 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.138430953, 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001444608, 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 935 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.174781+0000", 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.174781+0000", 2024-09-12T07:19:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.174782+0000", 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006e-06 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.174827+0000", 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5654e-05 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.174842+0000", 2024-09-12T07:19:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4401e-05 2024-09-12T07:19:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.174850+0000", 2024-09-12T07:19:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7000000000000008e-06 2024-09-12T07:19:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.174901+0000", 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1246999999999999e-05 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.175016+0000", 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011519300000000001 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.175124+0000", 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108268 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.176124+0000", 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0009993440000000001 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.176163+0000", 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9647000000000002e-05 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.176198+0000", 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4653000000000002e-05 2024-09-12T07:19:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.176225+0000", 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7495e-05 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:959 3.4 3:276183f2:::rbd_data.116011b0bca6.0000000000000058:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.228004+0000", 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.085207481000000002, 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081355130000000005, 2024-09-12T07:19:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 959 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.228004+0000", 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.228004+0000", 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.228006+0000", 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.086e-06 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.228016+0000", 2024-09-12T07:19:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.696e-06 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.228036+0000", 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9851999999999999e-05 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.228044+0000", 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8709999999999997e-06 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.228108+0000", 2024-09-12T07:19:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4239999999999995e-05 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.228174+0000", 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6136000000000005e-05 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.229583+0000", 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014085600000000001 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.235955+0000", 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063723349999999998 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236005+0000", 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9740000000000001e-05 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236081+0000", 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6128e-05 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236140+0000", 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8869000000000002e-05 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:959 3.4 e28/24 ondisk, result = 0)", 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.233410+0000", 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.079801431000000006, 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002736661, 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.233410+0000", 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.233410+0000", 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.233411+0000", 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4900000000000005e-07 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.233415+0000", 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4850000000000001e-06 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.233426+0000", 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1358999999999999e-05 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.233433+0000", 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0249999999999997e-06 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.235985+0000", 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025522740000000002 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.235997+0000", 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1381e-05 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236147+0000", 2024-09-12T07:19:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015028799999999999 2024-09-12T07:19:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.430 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:48.431 INFO:teuthology.orchestra.run.smithi046.stdout:398 punch from 0x6233eb9 to 0x6238f23, (0x506a bytes) 2024-09-12T07:19:48.431 INFO:teuthology.orchestra.run.smithi046.stdout:399 write 0x8da0ad1 thru 0x8da1436 (0x966 bytes) 2024-09-12T07:19:48.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:19:48.438 INFO:teuthology.orchestra.run.smithi046.stdout:400 write 0xe0ae898 thru 0xe0bdc4b (0xf3b4 bytes) 2024-09-12T07:19:48.438 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.438 INFO:teuthology.orchestra.run.smithi046.stdout:404 read 0xdd541f4 thru 0xdd58318 (0x4125 bytes) 2024-09-12T07:19:48.438 INFO:teuthology.orchestra.run.smithi046.stdout:406 write 0xe3568da thru 0xe358507 (0x1c2e bytes) 2024-09-12T07:19:48.438 INFO:teuthology.orchestra.run.smithi046.stdout:407 writesame 0xe310bf5 thru 0xe314b0d data_size 0x1(0x3f19 bytes) 2024-09-12T07:19:48.438 INFO:teuthology.orchestra.run.smithi046.stdout:408 read 0xa354bf thru 0xa3bc66 (0x67a8 bytes) 2024-09-12T07:19:48.438 INFO:teuthology.orchestra.run.smithi046.stdout:409 write 0x9edbde7 thru 0x9ee0562 (0x477c bytes) 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:410 trunc from 0xee5b241 to 0xd355cf6 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:411 write 0xbf84097 thru 0xbf9170e (0xd678 bytes) 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:413 punch from 0x24d3c42 to 0x24d4e1c, (0x11da bytes) 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:414 trunc from 0xd355cf6 to 0x6e04765 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:416 punch from 0x3a146f7 to 0x3a16179, (0x1a82 bytes) 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:417 punch from 0x177f90c to 0x1782c95, (0x3389 bytes) 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:418 read 0x418745 thru 0x4247de (0xc09a bytes) 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:420 writesame 0x7cec6e9 thru 0x7cf040d data_size 0x1(0x3d25 bytes) 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:422 read 0x4a1d925 thru 0x4a27f47 (0xa623 bytes) 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:423 punch from 0x61dc0d0 to 0x61dc45d, (0x38d bytes) 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:424 write 0x4d33438 thru 0x4d3da54 (0xa61d bytes) 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:425 writesame 0x12e1086 thru 0x12ec94d data_size 0xa2(0xb8c8 bytes) 2024-09-12T07:19:48.439 INFO:teuthology.orchestra.run.smithi046.stdout:427 write 0x3dd1dcf thru 0x3dde317 (0xc549 bytes) 2024-09-12T07:19:48.440 INFO:teuthology.orchestra.run.smithi046.stdout:428 write 0x8e974f8 thru 0x8e99ef3 (0x29fc bytes) 2024-09-12T07:19:48.440 INFO:teuthology.orchestra.run.smithi046.stdout:429 punch from 0x1d20da4 to 0x1d239b7, (0x2c13 bytes) 2024-09-12T07:19:48.440 INFO:teuthology.orchestra.run.smithi046.stdout:430 clone 2 order 23 su 524288 sc 8 2024-09-12T07:19:48.440 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:48.433+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:48.526 INFO:teuthology.orchestra.run.smithi046.stdout:leaving image image_client.0-clone1 intact 2024-09-12T07:19:48.533 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:48.534+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:48.588 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:19:48.588 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:48.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:19:48.634 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-12T07:19:48.634 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:48.635+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:48.710 INFO:teuthology.orchestra.run.smithi046.stdout:431 punch from 0x6418bfa to 0x6421242, (0x8648 bytes) 2024-09-12T07:19:48.711 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:48.711 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:48.711 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:19:48.711 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:19:48.711 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:48.713 INFO:teuthology.orchestra.run.smithi046.stdout:434 read 0x5234208 thru 0x5235f41 (0x1d3a bytes) 2024-09-12T07:19:48.719 INFO:teuthology.orchestra.run.smithi046.stdout:436 write 0x908a0ce thru 0x9097b5a (0xda8d bytes) 2024-09-12T07:19:48.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:19:48.725 INFO:teuthology.orchestra.run.smithi046.stdout:437 read 0x28e3d86 thru 0x28eb9ea (0x7c65 bytes) 2024-09-12T07:19:48.731 INFO:teuthology.orchestra.run.smithi046.stdout:438 trunc from 0x9097b5b to 0x389b58b 2024-09-12T07:19:48.734 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:48.736+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:48.773 INFO:teuthology.orchestra.run.smithi046.stdout:440 write 0x444f8af thru 0x4455f8a (0x66dc bytes) 2024-09-12T07:19:48.792 INFO:teuthology.orchestra.run.smithi046.stdout:442 write 0xbc4a785 thru 0xbc549d6 (0xa252 bytes) 2024-09-12T07:19:48.797 INFO:teuthology.orchestra.run.smithi046.stdout:443 write 0xd597 thru 0x14688 (0x70f2 bytes) 2024-09-12T07:19:48.799 INFO:teuthology.orchestra.run.smithi046.stdout:truncating to largest ever: 0xe2d5b7a 2024-09-12T07:19:48.802 INFO:teuthology.orchestra.run.smithi046.stdout:444 trunc from 0xbc549d7 to 0xe2d5b7a 2024-09-12T07:19:48.815 INFO:teuthology.orchestra.run.smithi046.stdout:446 trunc from 0xe2d5b7a to 0x6a7359c 2024-09-12T07:19:48.831 INFO:teuthology.orchestra.run.smithi046.stdout:448 write 0x2943533 thru 0x294c252 (0x8d20 bytes) 2024-09-12T07:19:48.835 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:48.837+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:48.841 INFO:teuthology.orchestra.run.smithi046.stdout:449 write 0x999fb26 thru 0x99aa8dc (0xadb7 bytes) 2024-09-12T07:19:48.844 INFO:teuthology.orchestra.run.smithi046.stdout:452 read 0x631d59a thru 0x6328c62 (0xb6c9 bytes) 2024-09-12T07:19:48.844 INFO:teuthology.orchestra.run.smithi046.stdout:456 read 0x85565f0 thru 0x855b770 (0x5181 bytes) 2024-09-12T07:19:48.844 INFO:teuthology.orchestra.run.smithi046.stdout:457 write 0x50f3b44 thru 0x50f7666 (0x3b23 bytes) 2024-09-12T07:19:48.847 INFO:teuthology.orchestra.run.smithi046.stdout:459 write 0x97fcdfd thru 0x9803f5c (0x7160 bytes) 2024-09-12T07:19:48.851 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:19:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:19:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:19:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4244.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-12T07:19:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:30.217471+0000", 2024-09-12T07:19:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.634206138, 2024-09-12T07:19:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014488159999999999, 2024-09-12T07:19:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217471+0000", 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217471+0000", 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217472+0000", 2024-09-12T07:19:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4200000000000002e-07 2024-09-12T07:19:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217482+0000", 2024-09-12T07:19:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0067e-05 2024-09-12T07:19:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217494+0000", 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2437e-05 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217522+0000", 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7929999999999999e-05 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.218022+0000", 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00049994099999999995 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.218103+0000", 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1038999999999994e-05 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.218872+0000", 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00076892799999999999 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.218920+0000", 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8131999999999999e-05 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.855 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:12 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.467367+0000", 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.3843104840000002, 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027275540000000001, 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.856 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467367+0000", 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467367+0000", 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467368+0000", 2024-09-12T07:19:48.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1519999999999999e-06 2024-09-12T07:19:48.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467372+0000", 2024-09-12T07:19:48.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1980000000000003e-06 2024-09-12T07:19:48.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467380+0000", 2024-09-12T07:19:48.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.515e-06 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467387+0000", 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0570000000000004e-06 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467437+0000", 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9738999999999999e-05 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:48.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467447+0000", 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0227999999999999e-05 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.468920+0000", 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014732059999999999 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469023+0000", 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103087 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469168+0000", 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014458099999999999 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469640+0000", 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047246499999999998 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470041+0000", 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040072399999999998 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470072+0000", 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1080000000000001e-05 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470095+0000", 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2521999999999999e-05 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.862 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:25 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [watch watch cookie 140539785655440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.478500+0000", 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.3731780440000003, 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014004029999999999, 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 25 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478500+0000", 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478500+0000", 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478500+0000", 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4000000000000001e-07 2024-09-12T07:19:48.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478504+0000", 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1250000000000003e-06 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478514+0000", 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3200000000000006e-06 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478518+0000", 2024-09-12T07:19:48.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478572+0000", 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3829999999999998e-05 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478658+0000", 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5891999999999997e-05 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:48.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.478828+0000", 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017019800000000001 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479385+0000", 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000556278 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479764+0000", 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037902999999999998 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479821+0000", 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7574000000000001e-05 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.479900+0000", 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8771999999999997e-05 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:492 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [watch unwatch cookie 140539785655440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.868 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.555907+0000", 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.2957700910000001, 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029325409999999999, 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 492 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555907+0000", 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555907+0000", 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555908+0000", 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4499999999999997e-07 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555912+0000", 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6150000000000001e-06 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555923+0000", 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0832000000000001e-05 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555930+0000", 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7279999999999995e-06 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556075+0000", 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014423300000000001 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556146+0000", 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1710000000000003e-05 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556235+0000", 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9161999999999998e-05 2024-09-12T07:19:48.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556559+0000", 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000323407 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558730+0000", 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021707190000000002 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558761+0000", 2024-09-12T07:19:48.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1504000000000002e-05 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558840+0000", 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8986000000000006e-05 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:493 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.556112+0000", 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.2955653709999999, 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029569990000000001, 2024-09-12T07:19:48.874 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 493 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556112+0000", 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556112+0000", 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556113+0000", 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6799999999999999e-07 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556117+0000", 2024-09-12T07:19:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2239999999999997e-06 2024-09-12T07:19:48.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556125+0000", 2024-09-12T07:19:48.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9799999999999998e-06 2024-09-12T07:19:48.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556129+0000", 2024-09-12T07:19:48.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1559999999999997e-06 2024-09-12T07:19:48.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556391+0000", 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026192499999999998 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for rw locks", 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556429+0000", 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7280000000000002e-05 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558887+0000", 2024-09-12T07:19:48.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024582580000000001 2024-09-12T07:19:48.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558980+0000", 2024-09-12T07:19:48.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3177999999999995e-05 2024-09-12T07:19:48.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.559069+0000", 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9229999999999998e-05 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:538 3.4 e28/24 3:220e9991:::rbd_data.116011b0bca6.000000000000002e:head v 28'1, mlcod=0'0)", 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.594295+0000", 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.2573822400000001, 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010840260000000001, 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.880 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.594295+0000", 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.594295+0000", 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.594296+0000", 2024-09-12T07:19:48.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1699999999999999e-07 2024-09-12T07:19:48.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.594347+0000", 2024-09-12T07:19:48.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1286999999999999e-05 2024-09-12T07:19:48.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.594360+0000", 2024-09-12T07:19:48.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3169999999999999e-05 2024-09-12T07:19:48.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.594370+0000", 2024-09-12T07:19:48.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0026e-05 2024-09-12T07:19:48.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.594429+0000", 2024-09-12T07:19:48.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8505000000000003e-05 2024-09-12T07:19:48.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.594453+0000", 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.425e-05 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.595335+0000", 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00088256100000000002 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.595379+0000", 2024-09-12T07:19:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3909999999999998e-05 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:583 3.4 e28/24 3:220e9991:::rbd_data.116011b0bca6.000000000000002e:head v 28'2, mlcod=0'0)", 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.668367+0000", 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.183310716, 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079772700000000003, 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668367+0000", 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668367+0000", 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668368+0000", 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3e-06 2024-09-12T07:19:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668393+0000", 2024-09-12T07:19:48.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4871e-05 2024-09-12T07:19:48.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668417+0000", 2024-09-12T07:19:48.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3552000000000001e-05 2024-09-12T07:19:48.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668426+0000", 2024-09-12T07:19:48.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0850000000000003e-06 2024-09-12T07:19:48.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668505+0000", 2024-09-12T07:19:48.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8931999999999996e-05 2024-09-12T07:19:48.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.668526+0000", 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0959e-05 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.669109+0000", 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00058296900000000002 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.669165+0000", 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6059e-05 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:610 3.9 3:911399f5:::rbd_data.116011b0bca6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.685417+0000", 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.166260356, 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001501474, 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 610 2024-09-12T07:19:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685417+0000", 2024-09-12T07:19:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685417+0000", 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685419+0000", 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.739e-06 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685427+0000", 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8890000000000002e-06 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685443+0000", 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6586999999999999e-05 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685450+0000", 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0729999999999999e-06 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685511+0000", 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0596000000000001e-05 2024-09-12T07:19:48.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685613+0000", 2024-09-12T07:19:48.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010168 2024-09-12T07:19:48.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:48.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685756+0000", 2024-09-12T07:19:48.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000143237 2024-09-12T07:19:48.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686751+0000", 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099451300000000004 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686802+0000", 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0998000000000002e-05 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686873+0000", 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1029e-05 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686919+0000", 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6133e-05 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:784 3.0 e28/24 3:067c6c85:::rbd_data.116011b0bca6.0000000000000024:head v 28'1, mlcod=0'0)", 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.946323+0000", 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.90535454000000004, 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010334439999999999, 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.897 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.946323+0000", 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.946323+0000", 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.946324+0000", 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.048e-06 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.946366+0000", 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2398999999999998e-05 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.946379+0000", 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2985e-05 2024-09-12T07:19:48.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.946386+0000", 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.438e-06 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.946444+0000", 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8591999999999997e-05 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.946471+0000", 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6279999999999999e-05 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.947320+0000", 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00084973100000000005 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.947356+0000", 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5970999999999998e-05 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:935 3.4 e28/24 3:276183f2:::rbd_data.116011b0bca6.0000000000000058:head v 28'4, mlcod=28'2)", 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.175215+0000", 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.67646249599999997, 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00083192900000000002, 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.175215+0000", 2024-09-12T07:19:48.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.175215+0000", 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.175216+0000", 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1570000000000001e-06 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.175264+0000", 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7339000000000002e-05 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.175283+0000", 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9221000000000001e-05 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.175293+0000", 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9150000000000008e-06 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.175358+0000", 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5466e-05 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.175376+0000", 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7448999999999998e-05 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.176002+0000", 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00062618799999999996 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.176047+0000", 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5194000000000001e-05 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:947 3.9 3:911399f5:::rbd_data.116011b0bca6.0000000000000007:head [stat,write 600105~38076] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.188058+0000", 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.66361914799999999, 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012781069999999999, 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 947 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.188058+0000", 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.188058+0000", 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.188061+0000", 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3240000000000001e-06 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.188083+0000", 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2480000000000002e-05 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.188104+0000", 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0988999999999998e-05 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.188116+0000", 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1395e-05 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.188181+0000", 2024-09-12T07:19:48.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5467000000000002e-05 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.188239+0000", 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7809999999999997e-05 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.188346+0000", 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010683699999999999 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.189082+0000", 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00073622599999999998 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.189273+0000", 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019073000000000001 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.189307+0000", 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4090000000000001e-05 2024-09-12T07:19:48.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.189337+0000", 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9759000000000001e-05 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:956 3.9 3:96f4f606:::rbd_data.116011b0bca6.0000000000000027:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.206153+0000", 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64552431600000004, 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016258261999999999, 2024-09-12T07:19:48.912 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 956 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206153+0000", 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206153+0000", 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206155+0000", 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0499999999999999e-06 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206163+0000", 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6699999999999994e-06 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206181+0000", 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7645000000000001e-05 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206188+0000", 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0829999999999998e-06 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206239+0000", 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0973000000000001e-05 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206346+0000", 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010729400000000001 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.209070+0000", 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027238340000000001 2024-09-12T07:19:48.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222329+0000", 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013259655 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222341+0000", 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.119e-05 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222378+0000", 2024-09-12T07:19:48.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7008000000000002e-05 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222411+0000", 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3859999999999998e-05 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:956 3.9 e28/24 ondisk, result = 0)", 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.217233+0000", 2024-09-12T07:19:48.918 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.63444466799999999, 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051019610000000003, 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217233+0000", 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217233+0000", 2024-09-12T07:19:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217234+0000", 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.579e-06 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217243+0000", 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3310000000000004e-06 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217258+0000", 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4715000000000001e-05 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217267+0000", 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1439999999999992e-06 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222316+0000", 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050498039999999998 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222325+0000", 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5480000000000002e-06 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222335+0000", 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8400000000000007e-06 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:959 3.4 e28/24 3:276183f2:::rbd_data.116011b0bca6.0000000000000058:head v 28'5, mlcod=28'3)", 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.231670+0000", 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.620007423, 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001049688, 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.231670+0000", 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.231670+0000", 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.231672+0000", 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.657e-06 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.231694+0000", 2024-09-12T07:19:48.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2172e-05 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.231712+0000", 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8244999999999999e-05 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.231722+0000", 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.231787+0000", 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4676000000000002e-05 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.231813+0000", 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6460000000000001e-05 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.232666+0000", 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00085249900000000005 2024-09-12T07:19:48.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.232720+0000", 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3903000000000002e-05 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:960 3.9 3:911399f5:::rbd_data.116011b0bca6.0000000000000007:head [stat,write 228792~62787] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.236487+0000", 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.61519064199999995, 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016742370000000001, 2024-09-12T07:19:48.927 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 960 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236487+0000", 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236487+0000", 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236489+0000", 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.686e-06 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236510+0000", 2024-09-12T07:19:48.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1060000000000002e-05 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236522+0000", 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.256e-05 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236528+0000", 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9150000000000001e-06 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236588+0000", 2024-09-12T07:19:48.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9398999999999998e-05 2024-09-12T07:19:48.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236692+0000", 2024-09-12T07:19:48.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104589 2024-09-12T07:19:48.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:48.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236857+0000", 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016502699999999999 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.237713+0000", 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00085576399999999996 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.238083+0000", 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037034999999999998 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.238115+0000", 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2211e-05 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.238161+0000", 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5676000000000001e-05 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:984 3.9 3:96f4f606:::rbd_data.116011b0bca6.0000000000000027:head [write 1013176~35400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.269836+0000", 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.58184141700000003, 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020110530000000001, 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 984 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269836+0000", 2024-09-12T07:19:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269836+0000", 2024-09-12T07:19:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269838+0000", 2024-09-12T07:19:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7770000000000001e-06 2024-09-12T07:19:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269857+0000", 2024-09-12T07:19:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9029999999999999e-05 2024-09-12T07:19:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269874+0000", 2024-09-12T07:19:48.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7359000000000001e-05 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269882+0000", 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269938+0000", 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5801999999999999e-05 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.270002+0000", 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4448000000000004e-05 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.270086+0000", 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3919999999999996e-05 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.270707+0000", 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00062067299999999997 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.271735+0000", 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010287720000000001 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.271796+0000", 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0791000000000001e-05 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.271847+0000", 2024-09-12T07:19:48.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1070999999999999e-05 2024-09-12T07:19:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:996 3.0 e28/24 3:05cac305:::rbd_data.116011b0bca6.0000000000000065:head v 28'3, mlcod=28'1)", 2024-09-12T07:19:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.314100+0000", 2024-09-12T07:19:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.53757755500000004, 2024-09-12T07:19:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00083715899999999997, 2024-09-12T07:19:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.314100+0000", 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.314100+0000", 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.314101+0000", 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0809999999999999e-06 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.314121+0000", 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0103e-05 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.314137+0000", 2024-09-12T07:19:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5543e-05 2024-09-12T07:19:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.314149+0000", 2024-09-12T07:19:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2473000000000001e-05 2024-09-12T07:19:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.314205+0000", 2024-09-12T07:19:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5838000000000003e-05 2024-09-12T07:19:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.314224+0000", 2024-09-12T07:19:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9201000000000001e-05 2024-09-12T07:19:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.314887+0000", 2024-09-12T07:19:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00066279000000000002 2024-09-12T07:19:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.314937+0000", 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0130000000000003e-05 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:1033 3.4 e28/24 3:276183f2:::rbd_data.116011b0bca6.0000000000000058:head v 28'6, mlcod=28'4)", 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.378030+0000", 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.47364762199999999, 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00085310400000000004, 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378030+0000", 2024-09-12T07:19:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378030+0000", 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378031+0000", 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.235e-06 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378044+0000", 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3038e-05 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378065+0000", 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0970000000000001e-05 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378076+0000", 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0515e-05 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378141+0000", 2024-09-12T07:19:48.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5322999999999998e-05 2024-09-12T07:19:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378166+0000", 2024-09-12T07:19:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5111999999999999e-05 2024-09-12T07:19:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378851+0000", 2024-09-12T07:19:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00068485400000000004 2024-09-12T07:19:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378883+0000", 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2057000000000002e-05 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1111 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.493764+0000", 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.35791334699999999, 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020530140000000001, 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.950 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1111 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493764+0000", 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493764+0000", 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493766+0000", 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6440000000000001e-06 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493774+0000", 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0749999999999998e-06 2024-09-12T07:19:48.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493791+0000", 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6799e-05 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493801+0000", 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9939999999999995e-06 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493859+0000", 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8269000000000001e-05 2024-09-12T07:19:48.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493864+0000", 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4970000000000001e-06 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.495607+0000", 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001742586 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.495670+0000", 2024-09-12T07:19:48.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3077000000000006e-05 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.495817+0000", 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014707300000000001 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:1126 3.0 e29/24 3:04479396:::rbd_id.image_client.0-clone2:head v 29'7, mlcod=28'5)", 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.502862+0000", 2024-09-12T07:19:48.955 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.34881564300000001, 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082778300000000001, 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.502862+0000", 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.502862+0000", 2024-09-12T07:19:48.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.502862+0000", 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.97e-07 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.502866+0000", 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4240000000000002e-06 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.502875+0000", 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.876e-06 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.502881+0000", 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3739999999999997e-06 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.502951+0000", 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0207000000000001e-05 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.502976+0000", 2024-09-12T07:19:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4641999999999999e-05 2024-09-12T07:19:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.503659+0000", 2024-09-12T07:19:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00068351900000000005 2024-09-12T07:19:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.503690+0000", 2024-09-12T07:19:48.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0543999999999998e-05 2024-09-12T07:19:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:48.960 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:48.960 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:48.961 INFO:teuthology.orchestra.run.smithi046.stdout:460 writesame 0x7733f91 thru 0x773d598 data_size 0x4(0x9608 bytes) 2024-09-12T07:19:48.961 INFO:teuthology.orchestra.run.smithi046.stdout:461 trunc from 0x99aa8dd to 0x8defefd 2024-09-12T07:19:48.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:19:48.966 INFO:teuthology.orchestra.run.smithi046.stdout:462 read 0x3bbedd2 thru 0x3bc7d28 (0x8f57 bytes) 2024-09-12T07:19:48.966 INFO:teuthology.orchestra.run.smithi046.stdout:463 punch from 0x1a8fdfc to 0x1a94b0a, (0x4d0e bytes) 2024-09-12T07:19:48.966 INFO:teuthology.orchestra.run.smithi046.stdout:465 trunc from 0x8defefd to 0x5b5f1aa 2024-09-12T07:19:48.967 INFO:teuthology.orchestra.run.smithi046.stdout:466 write 0x8c7e6a6 thru 0x8c80689 (0x1fe4 bytes) 2024-09-12T07:19:48.967 INFO:teuthology.orchestra.run.smithi046.stdout:467 read 0x41f3f9c thru 0x41f841e (0x4483 bytes) 2024-09-12T07:19:48.967 INFO:teuthology.orchestra.run.smithi046.stdout:468 read 0x7db7d42 thru 0x7dc2fff (0xb2be bytes) 2024-09-12T07:19:48.967 INFO:teuthology.orchestra.run.smithi046.stdout:469 read 0x3529375 thru 0x3535f07 (0xcb93 bytes) 2024-09-12T07:19:48.967 INFO:teuthology.orchestra.run.smithi046.stdout:470 write 0x5136bf2 thru 0x513ca3b (0x5e4a bytes) 2024-09-12T07:19:48.967 INFO:teuthology.orchestra.run.smithi046.stdout:471 trunc from 0x8c8068a to 0x47e7336 2024-09-12T07:19:48.967 INFO:teuthology.orchestra.run.smithi046.stdout:472 writesame 0x925afe3 thru 0x9264845 data_size 0x1(0x9863 bytes) 2024-09-12T07:19:48.968 INFO:teuthology.orchestra.run.smithi046.stdout:473 writesame 0xb193255 thru 0xb1a1c71 data_size 0x1(0xea1d bytes) 2024-09-12T07:19:48.968 INFO:teuthology.orchestra.run.smithi046.stdout:474 read 0x41ee9e7 thru 0x41fe08a (0xf6a4 bytes) 2024-09-12T07:19:48.968 INFO:teuthology.orchestra.run.smithi046.stdout:475 read 0x93ce3a6 thru 0x93d1428 (0x3083 bytes) 2024-09-12T07:19:48.968 INFO:teuthology.orchestra.run.smithi046.stdout:476 trunc from 0xb1a1c72 to 0x458f97f 2024-09-12T07:19:48.968 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:48.968 INFO:teuthology.orchestra.run.smithi046.stdout:480 read 0x1da258a thru 0x1da7502 (0x4f79 bytes) 2024-09-12T07:19:48.969 INFO:teuthology.orchestra.run.smithi046.stdout:481 punch from 0x346f794 to 0x347c0db, (0xc947 bytes) 2024-09-12T07:19:48.969 INFO:teuthology.orchestra.run.smithi046.stdout:482 read 0x9bd49c thru 0x9c2c0d (0x5772 bytes) 2024-09-12T07:19:48.969 INFO:teuthology.orchestra.run.smithi046.stdout:484 writesame 0x5ca5dc6 thru 0x5caae50 data_size 0x4f(0x508b bytes) 2024-09-12T07:19:48.969 INFO:teuthology.orchestra.run.smithi046.stdout:485 punch from 0x5a02e98 to 0x5a0ed07, (0xbe6f bytes) 2024-09-12T07:19:48.969 INFO:teuthology.orchestra.run.smithi046.stdout:486 punch from 0x323d2f2 to 0x3249c8b, (0xc999 bytes) 2024-09-12T07:19:48.970 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:48.963+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:48.971 INFO:teuthology.orchestra.run.smithi046.stdout:489 write 0x78a759c thru 0x78b4c55 (0xd6ba bytes) 2024-09-12T07:19:48.978 INFO:teuthology.orchestra.run.smithi046.stdout:490 write 0x8286b3a thru 0x828bf87 (0x544e bytes) 2024-09-12T07:19:48.989 INFO:teuthology.orchestra.run.smithi046.stdout:491 trunc from 0x828bf88 to 0xbb8a68 2024-09-12T07:19:49.019 INFO:teuthology.orchestra.run.smithi046.stdout:493 write 0xb87ab30 thru 0xb880bff (0x60d0 bytes) 2024-09-12T07:19:49.023 INFO:teuthology.orchestra.run.smithi046.stdout:495 read 0x5b78dbb thru 0x5b7d576 (0x47bc bytes) 2024-09-12T07:19:49.023 INFO:teuthology.orchestra.run.smithi046.stdout:496 clone 3 order 25 su 2097152 sc 4 2024-09-12T07:19:49.062 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:49.064+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:49.082 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:19:49.082 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:49.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:19:49.174 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:49.176+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:49.238 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:49.238 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:49.238 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:19:49.238 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:19:49.238 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:49.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:19:49.291 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:49.292+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:49.364 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:49.366+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:49.399 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:49.399 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:19:49.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:19:49.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:19:49.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.399 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:611 3.8 3:10500085:::rbd_data.116011b0bca6.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1163555] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:49.399 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.685767+0000", 2024-09-12T07:19:49.399 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.7135446480000001, 2024-09-12T07:19:49.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002214756, 2024-09-12T07:19:49.399 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 611 2024-09-12T07:19:49.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.400 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685767+0000", 2024-09-12T07:19:49.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685767+0000", 2024-09-12T07:19:49.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685769+0000", 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.578e-06 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685776+0000", 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3900000000000004e-06 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685791+0000", 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4819e-05 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685799+0000", 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1920000000000005e-06 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685870+0000", 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1418e-05 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686050+0000", 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179228 2024-09-12T07:19:49.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-12T07:19:49.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686263+0000", 2024-09-12T07:19:49.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021295299999999999 2024-09-12T07:19:49.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687803+0000", 2024-09-12T07:19:49.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001540502 2024-09-12T07:19:49.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687870+0000", 2024-09-12T07:19:49.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7206999999999995e-05 2024-09-12T07:19:49.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687928+0000", 2024-09-12T07:19:49.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7340999999999999e-05 2024-09-12T07:19:49.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687982+0000", 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4128000000000001e-05 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:613 3.7 3:e175feac:::rbd_data.116011b0bca6.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.686095+0000", 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.7132161379999999, 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054618599999999998, 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 613 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686095+0000", 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686095+0000", 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686097+0000", 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0699999999999999e-06 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686102+0000", 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6570000000000002e-06 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686114+0000", 2024-09-12T07:19:49.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1637999999999999e-05 2024-09-12T07:19:49.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686123+0000", 2024-09-12T07:19:49.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6789999999999994e-06 2024-09-12T07:19:49.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686187+0000", 2024-09-12T07:19:49.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4734999999999996e-05 2024-09-12T07:19:49.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686337+0000", 2024-09-12T07:19:49.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000149383 2024-09-12T07:19:49.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686544+0000", 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020754000000000001 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687923+0000", 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013792850000000001 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691480+0000", 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035560660000000001 2024-09-12T07:19:49.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691514+0000", 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3979000000000001e-05 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691557+0000", 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3828000000000001e-05 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:615 3.8 3:14ff501c:::rbd_data.116011b0bca6.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.686346+0000", 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.7129660869999999, 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021688020000000001, 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 615 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686346+0000", 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686346+0000", 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686347+0000", 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.029e-06 2024-09-12T07:19:49.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686371+0000", 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4538000000000001e-05 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686394+0000", 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2770000000000001e-05 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686402+0000", 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2630000000000003e-06 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686538+0000", 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013633100000000001 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686650+0000", 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011142 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686843+0000", 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019292300000000001 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687996+0000", 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001153292 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.688428+0000", 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043179200000000002 2024-09-12T07:19:49.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.688471+0000", 2024-09-12T07:19:49.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2970999999999999e-05 2024-09-12T07:19:49.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.688514+0000", 2024-09-12T07:19:49.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3473000000000002e-05 2024-09-12T07:19:49.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.420 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.420 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:617 3.2 3:4522120e:::rbd_data.116011b0bca6.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:49.421 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.687024+0000", 2024-09-12T07:19:49.421 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.712287302, 2024-09-12T07:19:49.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047272190000000004, 2024-09-12T07:19:49.421 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 617 2024-09-12T07:19:49.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.422 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687024+0000", 2024-09-12T07:19:49.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687024+0000", 2024-09-12T07:19:49.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687025+0000", 2024-09-12T07:19:49.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6000000000000002e-07 2024-09-12T07:19:49.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687055+0000", 2024-09-12T07:19:49.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.029e-05 2024-09-12T07:19:49.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687065+0000", 2024-09-12T07:19:49.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6749999999999997e-06 2024-09-12T07:19:49.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687072+0000", 2024-09-12T07:19:49.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.657e-06 2024-09-12T07:19:49.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687131+0000", 2024-09-12T07:19:49.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9157000000000003e-05 2024-09-12T07:19:49.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687263+0000", 2024-09-12T07:19:49.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013171799999999999 2024-09-12T07:19:49.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:19:49.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687458+0000", 2024-09-12T07:19:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019529899999999999 2024-09-12T07:19:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.688377+0000", 2024-09-12T07:19:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00091874700000000003 2024-09-12T07:19:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691642+0000", 2024-09-12T07:19:49.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0032655290000000001 2024-09-12T07:19:49.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691706+0000", 2024-09-12T07:19:49.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3548999999999998e-05 2024-09-12T07:19:49.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691752+0000", 2024-09-12T07:19:49.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5738000000000002e-05 2024-09-12T07:19:49.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.433 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.434 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:894 3.2 3:468e8e09:::rbd_data.116011b0bca6.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:49.434 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:19:49.434 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.296399694, 2024-09-12T07:19:49.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015762987999999999, 2024-09-12T07:19:49.434 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.435 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 894 2024-09-12T07:19:49.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.435 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:19:49.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:19:49.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102913+0000", 2024-09-12T07:19:49.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004e-06 2024-09-12T07:19:49.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102918+0000", 2024-09-12T07:19:49.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6260000000000003e-06 2024-09-12T07:19:49.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102929+0000", 2024-09-12T07:19:49.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1382e-05 2024-09-12T07:19:49.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102934+0000", 2024-09-12T07:19:49.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:19:49.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102988+0000", 2024-09-12T07:19:49.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3683000000000001e-05 2024-09-12T07:19:49.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.103072+0000", 2024-09-12T07:19:49.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4135000000000007e-05 2024-09-12T07:19:49.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:19:49.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.105687+0000", 2024-09-12T07:19:49.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026149580000000001 2024-09-12T07:19:49.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118570+0000", 2024-09-12T07:19:49.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012883034 2024-09-12T07:19:49.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118585+0000", 2024-09-12T07:19:49.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4797000000000001e-05 2024-09-12T07:19:49.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118622+0000", 2024-09-12T07:19:49.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7163000000000003e-05 2024-09-12T07:19:49.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118675+0000", 2024-09-12T07:19:49.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3236000000000003e-05 2024-09-12T07:19:49.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.446 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.447 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:894 3.2 e28/24 ondisk, result = 0)", 2024-09-12T07:19:49.447 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.112622+0000", 2024-09-12T07:19:49.447 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.2866901159999999, 2024-09-12T07:19:49.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059558010000000002, 2024-09-12T07:19:49.447 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.448 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:49.448 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112622+0000", 2024-09-12T07:19:49.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112622+0000", 2024-09-12T07:19:49.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112623+0000", 2024-09-12T07:19:49.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9800000000000002e-07 2024-09-12T07:19:49.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112627+0000", 2024-09-12T07:19:49.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0720000000000001e-06 2024-09-12T07:19:49.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112640+0000", 2024-09-12T07:19:49.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3131e-05 2024-09-12T07:19:49.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112649+0000", 2024-09-12T07:19:49.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8710000000000003e-06 2024-09-12T07:19:49.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118557+0000", 2024-09-12T07:19:49.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059084740000000004 2024-09-12T07:19:49.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118565+0000", 2024-09-12T07:19:49.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5270000000000001e-06 2024-09-12T07:19:49.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118577+0000", 2024-09-12T07:19:49.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2728000000000001e-05 2024-09-12T07:19:49.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.455 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.456 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:912 3.2 3:468e8e09:::rbd_data.116011b0bca6.000000000000003e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:49.456 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.143632+0000", 2024-09-12T07:19:49.456 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.255679413, 2024-09-12T07:19:49.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051497230000000001, 2024-09-12T07:19:49.457 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.457 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.457 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.457 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.457 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 912 2024-09-12T07:19:49.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.458 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143632+0000", 2024-09-12T07:19:49.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143632+0000", 2024-09-12T07:19:49.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143634+0000", 2024-09-12T07:19:49.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0040000000000002e-06 2024-09-12T07:19:49.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143641+0000", 2024-09-12T07:19:49.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1199999999999996e-06 2024-09-12T07:19:49.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143660+0000", 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8408999999999999e-05 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143670+0000", 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0312999999999999e-05 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143747+0000", 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7399999999999998e-05 2024-09-12T07:19:49.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143855+0000", 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010739999999999999 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.144004+0000", 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014922500000000001 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.144699+0000", 2024-09-12T07:19:49.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00069510800000000005 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.148690+0000", 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039904490000000001 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.148741+0000", 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1328000000000001e-05 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.148782+0000", 2024-09-12T07:19:49.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:993 3.7 3:e61faebc:::rbd_data.116011b0bca6.000000000000004c:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.295319+0000", 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1039928059999999, 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079320569999999993, 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 993 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295319+0000", 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295319+0000", 2024-09-12T07:19:49.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295321+0000", 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7e-06 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295328+0000", 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2729999999999997e-06 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295344+0000", 2024-09-12T07:19:49.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6033e-05 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295353+0000", 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2699999999999993e-06 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295409+0000", 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5507000000000002e-05 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295521+0000", 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112719 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.296644+0000", 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001123045 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303082+0000", 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064377510000000002 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303131+0000", 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9295999999999997e-05 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303195+0000", 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4022999999999993e-05 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303251+0000", 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5439999999999998e-05 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.469 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:993 3.7 e28/24 ondisk, result = 0)", 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.300433+0000", 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.0988781940000001, 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002823866, 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.300433+0000", 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.300433+0000", 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.300434+0000", 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4e-07 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.300438+0000", 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.304e-06 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.300446+0000", 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8310000000000007e-06 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.300452+0000", 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9070000000000004e-06 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303112+0000", 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026600349999999998 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303123+0000", 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1018e-05 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303257+0000", 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000133931 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1031 3.7 3:e61faebc:::rbd_data.116011b0bca6.000000000000004c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.377551+0000", 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.021760612, 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050361379999999999, 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1031 2024-09-12T07:19:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377551+0000", 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377551+0000", 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377554+0000", 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7970000000000001e-06 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377565+0000", 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1612e-05 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377589+0000", 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3955999999999999e-05 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377603+0000", 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3540000000000001e-05 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377698+0000", 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5172000000000003e-05 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377793+0000", 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5137999999999996e-05 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377920+0000", 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126518 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378626+0000", 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00070584200000000004 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.382513+0000", 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038871539999999999 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.382542+0000", 2024-09-12T07:19:49.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9583000000000001e-05 2024-09-12T07:19:49.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.382587+0000", 2024-09-12T07:19:49.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4826000000000001e-05 2024-09-12T07:19:49.479 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1080 3.b 3:d0645053:::rbd_object_map.116011b0bca6.0000000000000004:head [writefull 0~53] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.470376+0000", 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.92893561400000002, 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029172759999999999, 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.480 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1080 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470376+0000", 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470376+0000", 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470379+0000", 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.571e-06 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470388+0000", 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1279999999999997e-06 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470405+0000", 2024-09-12T07:19:49.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7229000000000001e-05 2024-09-12T07:19:49.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470416+0000", 2024-09-12T07:19:49.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0781000000000001e-05 2024-09-12T07:19:49.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470489+0000", 2024-09-12T07:19:49.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3366000000000002e-05 2024-09-12T07:19:49.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:49.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470497+0000", 2024-09-12T07:19:49.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0180000000000001e-06 2024-09-12T07:19:49.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.472041+0000", 2024-09-12T07:19:49.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015442780000000001 2024-09-12T07:19:49.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.472118+0000", 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6599000000000003e-05 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.472198+0000", 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9742999999999999e-05 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.472726+0000", 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052847500000000004 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.473232+0000", 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00050537100000000001 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.473262+0000", 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0827999999999997e-05 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.473293+0000", 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0889000000000002e-05 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1143 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.514700+0000", 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.88461116100000003, 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029554020000000002, 2024-09-12T07:19:49.487 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1143 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514700+0000", 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514700+0000", 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514701+0000", 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2900000000000003e-07 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514705+0000", 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8260000000000003e-06 2024-09-12T07:19:49.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514713+0000", 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3440000000000001e-06 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514717+0000", 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9500000000000003e-06 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514757+0000", 2024-09-12T07:19:49.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9672999999999998e-05 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514799+0000", 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1978999999999999e-05 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514927+0000", 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012807000000000001 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.515330+0000", 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040297199999999998 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.517605+0000", 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022750610000000001 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.517634+0000", 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.853e-05 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.517656+0000", 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2268000000000001e-05 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1338 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.748194+0000", 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.65111736399999998, 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038514669999999999, 2024-09-12T07:19:49.493 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1338 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748194+0000", 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748194+0000", 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748196+0000", 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4759999999999999e-06 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748647+0000", 2024-09-12T07:19:49.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00045091599999999999 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748661+0000", 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4534e-05 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748668+0000", 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9009999999999997e-06 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748723+0000", 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4468000000000001e-05 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748830+0000", 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010779 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748996+0000", 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165162 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.751387+0000", 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002391706 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.751907+0000", 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051961499999999996 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.751981+0000", 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4319000000000005e-05 2024-09-12T07:19:49.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.752046+0000", 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4579999999999995e-05 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1343 3.7 3:e78f2392:::rbd_data.116075042ea1.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7340032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.755402+0000", 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64390937699999995, 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0040308560000000002, 2024-09-12T07:19:49.499 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1343 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755402+0000", 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755402+0000", 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755405+0000", 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.937e-06 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755456+0000", 2024-09-12T07:19:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0642000000000001e-05 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755479+0000", 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3527e-05 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755492+0000", 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3132999999999999e-05 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755569+0000", 2024-09-12T07:19:49.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6323999999999996e-05 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755722+0000", 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000153394 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755924+0000", 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020157500000000001 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.757983+0000", 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00205894 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759317+0000", 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013344229999999999 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759382+0000", 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4969999999999996e-05 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759433+0000", 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0991e-05 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1344 3.18 3:1a9d4b56:::rbd_data.116075042ea1.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7340032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.755621+0000", 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.6436906, 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0040014960000000002, 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.505 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1344 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755621+0000", 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755621+0000", 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755622+0000", 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2600000000000001e-07 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755847+0000", 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022498 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755858+0000", 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1426e-05 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755866+0000", 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6559999999999992e-06 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755973+0000", 2024-09-12T07:19:49.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010666300000000001 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756091+0000", 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011859800000000001 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756678+0000", 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00058683100000000001 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759291+0000", 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002612582 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759551+0000", 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026036000000000001 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759585+0000", 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3516999999999998e-05 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759623+0000", 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7957000000000003e-05 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:1341 3.19 e29/24 3:98c9a0e4:::rbd_data.116075042ea1.0000000000000004:head v 29'99, mlcod=28'97)", 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.755920+0000", 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64339122199999998, 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031212290000000001, 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755920+0000", 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755920+0000", 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755921+0000", 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6699999999999999e-07 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755950+0000", 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9479999999999999e-05 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755961+0000", 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0352e-05 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755968+0000", 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0890000000000002e-06 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756047+0000", 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9684000000000005e-05 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756069+0000", 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1795999999999999e-05 2024-09-12T07:19:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.758791+0000", 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002721472 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759042+0000", 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00025088900000000002 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.515 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1461 3.7 3:e1f4f01d:::rbd_data.116075042ea1.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write 2090662~6490] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.894307+0000", 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.50500490600000003, 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002351372, 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1461 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.894307+0000", 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.894307+0000", 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.894308+0000", 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0599999999999999e-07 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.894316+0000", 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4339999999999992e-06 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.894327+0000", 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0915e-05 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.894332+0000", 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3990000000000003e-06 2024-09-12T07:19:49.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.894376+0000", 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4088999999999997e-05 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.894426+0000", 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9576000000000001e-05 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.894510+0000", 2024-09-12T07:19:49.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4052000000000001e-05 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.895167+0000", 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00065695800000000002 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.896602+0000", 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014355360000000001 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.896634+0000", 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1167000000000002e-05 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.896658+0000", 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.444e-05 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1472 3.2 3:42bbf481:::rbd_data.116075042ea1.000000000000000a:head [truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:49.521 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.910430+0000", 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.48888204800000001, 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0023851789999999999, 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1472 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.910430+0000", 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.910430+0000", 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.910431+0000", 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.910434+0000", 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6260000000000001e-06 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.910444+0000", 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8369999999999992e-06 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.910450+0000", 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.910506+0000", 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6595999999999998e-05 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.910591+0000", 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4778999999999996e-05 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.910726+0000", 2024-09-12T07:19:49.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013478200000000001 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.911300+0000", 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00057387200000000001 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.912733+0000", 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001433225 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.912772+0000", 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9196000000000002e-05 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.912815+0000", 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2729999999999999e-05 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1554 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:head [stat,truncate 1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.997429+0000", 2024-09-12T07:19:49.527 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.40188259399999998, 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002137351, 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1554 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997429+0000", 2024-09-12T07:19:49.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997429+0000", 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997430+0000", 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6099999999999999e-07 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997434+0000", 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2270000000000004e-06 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997446+0000", 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1581e-05 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997453+0000", 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4919999999999996e-06 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997516+0000", 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2393999999999999e-05 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997605+0000", 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9783000000000004e-05 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997746+0000", 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014089500000000001 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.998864+0000", 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011174889999999999 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999466+0000", 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00060243899999999997 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999522+0000", 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5955999999999998e-05 2024-09-12T07:19:49.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999566+0000", 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4134000000000001e-05 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1559 3.7 3:e78f2392:::rbd_data.116075042ea1.0000000000000005:head [stat,truncate 1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.997522+0000", 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.40179005299999998, 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021506659999999999, 2024-09-12T07:19:49.533 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1559 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997522+0000", 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997522+0000", 2024-09-12T07:19:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997523+0000", 2024-09-12T07:19:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0890000000000001e-06 2024-09-12T07:19:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997527+0000", 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8040000000000001e-06 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997555+0000", 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7291999999999999e-05 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997563+0000", 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3869999999999995e-06 2024-09-12T07:19:49.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997625+0000", 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2220999999999999e-05 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997708+0000", 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3125000000000001e-05 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997840+0000", 2024-09-12T07:19:49.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013145600000000001 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999090+0000", 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012497400000000001 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999586+0000", 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00049606599999999998 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:49.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999631+0000", 2024-09-12T07:19:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4857000000000001e-05 2024-09-12T07:19:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999672+0000", 2024-09-12T07:19:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1628999999999997e-05 2024-09-12T07:19:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:49.539 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:49.540 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:19:49.541 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:19:49.542 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:49.542+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:49.545 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone2 from 0xb880c00 (overlap 0xbb8a68) to 0x4b0429 2024-09-12T07:19:49.626 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-12T07:19:49.642 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:49.643+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:49.657 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:19:49.658 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:49.664 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:19:49.668 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:19:49.742 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:49.744+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:49.786 INFO:teuthology.orchestra.run.smithi046.stdout:497 read 0x12f4703 thru 0x12fe421 (0x9d1f bytes) 2024-09-12T07:19:49.797 INFO:teuthology.orchestra.run.smithi046.stdout:500 writesame 0xcb0b69a thru 0xcb13d59 data_size 0x40(0x86c0 bytes) 2024-09-12T07:19:49.800 INFO:teuthology.orchestra.run.smithi046.stdout:502 read 0x9fb7764 thru 0x9fbb5b3 (0x3e50 bytes) 2024-09-12T07:19:49.800 INFO:teuthology.orchestra.run.smithi046.stdout:504 write 0x57b0b76 thru 0x57bd1a0 (0xc62b bytes) 2024-09-12T07:19:49.806 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:19:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:19:49.807 INFO:teuthology.orchestra.run.smithi046.stdout:505 read 0x5dfefad thru 0x5e0496f (0x59c3 bytes) 2024-09-12T07:19:49.807 INFO:teuthology.orchestra.run.smithi046.stdout:506 read 0x37abcc9 thru 0x37b58b9 (0x9bf1 bytes) 2024-09-12T07:19:49.808 INFO:teuthology.orchestra.run.smithi046.stdout:508 writesame 0x1b6ba3c thru 0x1b74c85 data_size 0x46(0x924a bytes) 2024-09-12T07:19:49.813 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:49.813 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:49.813 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:19:49.813 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:19:49.813 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:49.814 INFO:teuthology.orchestra.run.smithi046.stdout:509 write 0x7f68c78 thru 0x7f6c874 (0x3bfd bytes) 2024-09-12T07:19:49.816 INFO:teuthology.orchestra.run.smithi046.stdout:510 trunc from 0xcb13d5a to 0xc67cae4 2024-09-12T07:19:49.823 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:19:49.829 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:19:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:19:49.840 INFO:teuthology.orchestra.run.smithi046.stdout:511 trunc from 0xc67cae4 to 0x11867ac 2024-09-12T07:19:49.850 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:19:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:19:49.867 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:49.869+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:49.868 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:49.868 INFO:teuthology.orchestra.run.smithi046.stdout:515 read 0x275252 thru 0x27ec37 (0x99e6 bytes) 2024-09-12T07:19:49.868 INFO:teuthology.orchestra.run.smithi046.stdout:516 read 0x916463 thru 0x918c18 (0x27b6 bytes) 2024-09-12T07:19:49.869 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:19:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:19:49.882 INFO:teuthology.orchestra.run.smithi046.stdout:518 write 0x780e562 thru 0x781a8b0 (0xc34f bytes) 2024-09-12T07:19:49.887 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:19:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:19:49.894 INFO:teuthology.orchestra.run.smithi046.stdout:520 write 0xc4004eb thru 0xc4076c9 (0x71df bytes) 2024-09-12T07:19:49.905 INFO:teuthology.orchestra.run.smithi046.stdout:522 trunc from 0xc4076ca to 0x55a41ca 2024-09-12T07:19:49.905 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:19:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:19:49.919 INFO:teuthology.orchestra.run.smithi046.stdout:523 writesame 0x4cbc7ce thru 0x4cca233 data_size 0x5(0xda66 bytes) 2024-09-12T07:19:49.923 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:19:49.923 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:19:49.925 INFO:teuthology.orchestra.run.smithi046.stdout:524 write 0x27c4051 thru 0x27c6886 (0x2836 bytes) 2024-09-12T07:19:49.926 INFO:teuthology.orchestra.run.smithi046.stdout:526 read 0x10537ce thru 0x1056578 (0x2dab bytes) 2024-09-12T07:19:49.927 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:49.927 INFO:teuthology.orchestra.run.smithi046.stdout:529 punch from 0x1af151d to 0x1afbff9, (0xaadc bytes) 2024-09-12T07:19:49.927 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:49.927 INFO:teuthology.orchestra.run.smithi046.stdout:531 punch from 0x52ad715 to 0x52b8cad, (0xb598 bytes) 2024-09-12T07:19:49.934 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:49.934 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:49.935 INFO:teuthology.orchestra.run.smithi046.stdout:532 trunc from 0x55a41ca to 0xca589f0 2024-09-12T07:19:49.940 INFO:teuthology.orchestra.run.smithi046.stdout:533 write 0xc9cbd1d thru 0xc9d33bf (0x76a3 bytes) 2024-09-12T07:19:49.941 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:19:49.941 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:19:49.943 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:19:49.944 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:49.945+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:49.944 INFO:teuthology.orchestra.run.smithi046.stdout:534 read 0x96c93aa thru 0x96cc5bc (0x3213 bytes) 2024-09-12T07:19:49.944 INFO:teuthology.orchestra.run.smithi046.stdout:535 write 0x3c1304f thru 0x3c1bb8a (0x8b3c bytes) 2024-09-12T07:19:49.960 INFO:teuthology.orchestra.run.smithi046.stdout:536 trunc from 0xca589f0 to 0xadc108 2024-09-12T07:19:49.974 INFO:teuthology.orchestra.run.smithi046.stdout:538 read 0x13f626 thru 0x14221b (0x2bf6 bytes) 2024-09-12T07:19:49.990 INFO:teuthology.orchestra.run.smithi046.stdout:542 writesame 0x993e3fd thru 0x9945789 data_size 0x1(0x738d bytes) 2024-09-12T07:19:49.990 INFO:teuthology.orchestra.run.smithi046.stdout:543 punch from 0xa3cdb to 0xa41c0, (0x4e5 bytes) 2024-09-12T07:19:49.990 INFO:teuthology.orchestra.run.smithi046.stdout:545 write 0x2e96d7 thru 0x2f04a7 (0x6dd1 bytes) 2024-09-12T07:19:49.991 INFO:teuthology.orchestra.run.smithi046.stdout:546 read 0x6db36cf thru 0x6dc134b (0xdc7d bytes) 2024-09-12T07:19:49.991 INFO:teuthology.orchestra.run.smithi046.stdout:547 read 0x55b5b7f thru 0x55b9cef (0x4171 bytes) 2024-09-12T07:19:50.001 INFO:teuthology.orchestra.run.smithi046.stdout:548 write 0xdf05ee8 thru 0xdf0ae63 (0x4f7c bytes) 2024-09-12T07:19:50.004 INFO:teuthology.orchestra.run.smithi046.stdout:549 punch from 0xc60a4a5 to 0xc60e904, (0x445f bytes) 2024-09-12T07:19:50.004 INFO:teuthology.orchestra.run.smithi046.stdout:550 write 0x85bdc14 thru 0x85c4fd2 (0x73bf bytes) 2024-09-12T07:19:50.007 INFO:teuthology.orchestra.run.smithi046.stdout:551 writesame 0x65a573b thru 0x65a6e0b data_size 0xb(0x16d1 bytes) 2024-09-12T07:19:50.008 INFO:teuthology.orchestra.run.smithi046.stdout:552 read 0xb98be0e thru 0xb9949e1 (0x8bd4 bytes) 2024-09-12T07:19:50.008 INFO:teuthology.orchestra.run.smithi046.stdout:553 writesame 0x77b21fa thru 0x77bd364 data_size 0x1(0xb16b bytes) 2024-09-12T07:19:50.032 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:50.045 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:50.046+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:50.045 INFO:teuthology.orchestra.run.smithi046.stdout:556 trunc from 0xdf0ae64 to 0x26dc55d 2024-09-12T07:19:50.057 INFO:teuthology.orchestra.run.smithi046.stdout:558 write 0x1a3f756 thru 0x1a4f375 (0xfc20 bytes) 2024-09-12T07:19:50.058 INFO:teuthology.orchestra.run.smithi046.stdout:559 punch from 0x1156545 to 0x1160f21, (0xa9dc bytes) 2024-09-12T07:19:50.058 INFO:teuthology.orchestra.run.smithi046.stdout:560 punch from 0xc9a0d0 to 0xca5caf, (0xbbdf bytes) 2024-09-12T07:19:50.076 INFO:teuthology.orchestra.run.smithi046.stdout:561 write 0xd7081d0 thru 0xd71533b (0xd16c bytes) 2024-09-12T07:19:50.079 INFO:teuthology.orchestra.run.smithi046.stdout:563 trunc from 0xd71533c to 0xdca0781 2024-09-12T07:19:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:565 read 0x9a13521 thru 0x9a19ffd (0x6add bytes) 2024-09-12T07:19:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:566 read 0x8d68bbd thru 0x8d6e572 (0x59b6 bytes) 2024-09-12T07:19:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:567 write 0x16b3de8 thru 0x16c2fc4 (0xf1dd bytes) 2024-09-12T07:19:50.086 INFO:teuthology.orchestra.run.smithi046.stdout:568 writesame 0x6de935e thru 0x6df6641 data_size 0x2c(0xd2e4 bytes) 2024-09-12T07:19:50.087 INFO:teuthology.orchestra.run.smithi046.stdout:569 read 0x1e6ab3c thru 0x1e727b1 (0x7c76 bytes) 2024-09-12T07:19:50.088 INFO:teuthology.orchestra.run.smithi046.stdout:570 write 0x8d282a3 thru 0x8d2c2b6 (0x4014 bytes) 2024-09-12T07:19:50.096 INFO:teuthology.orchestra.run.smithi046.stdout:571 trunc from 0xdca0781 to 0x93f4922 2024-09-12T07:19:50.099 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:50.100 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:50.102 INFO:teuthology.orchestra.run.smithi046.stdout:573 write 0x176e4fc thru 0x176ebd6 (0x6db bytes) 2024-09-12T07:19:50.104 INFO:teuthology.orchestra.run.smithi046.stdout:574 write 0x455d464 thru 0x456429c (0x6e39 bytes) 2024-09-12T07:19:50.106 INFO:teuthology.orchestra.run.smithi046.stdout:575 writesame 0x2d2f710 thru 0x2d3df82 data_size 0x1(0xe873 bytes) 2024-09-12T07:19:50.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:19:50.134 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:50.134 INFO:teuthology.orchestra.run.smithi046.stdout:577 read 0x5f5af39 thru 0x5f5e9ae (0x3a76 bytes) 2024-09-12T07:19:50.134 INFO:teuthology.orchestra.run.smithi046.stdout:578 punch from 0x18031af to 0x1811285, (0xe0d6 bytes) 2024-09-12T07:19:50.135 INFO:teuthology.orchestra.run.smithi046.stdout:579 punch from 0x710988b to 0x710c618, (0x2d8d bytes) 2024-09-12T07:19:50.135 INFO:teuthology.orchestra.run.smithi046.stdout:581 punch from 0x6c3adc0 to 0x6c46dd5, (0xc015 bytes) 2024-09-12T07:19:50.140 INFO:teuthology.orchestra.run.smithi046.stdout:582 writesame 0x973e092 thru 0x974b731 data_size 0x88(0xd6a0 bytes) 2024-09-12T07:19:50.141 INFO:teuthology.orchestra.run.smithi046.stdout:583 read 0x54f214d thru 0x5501d50 (0xfc04 bytes) 2024-09-12T07:19:50.143 INFO:teuthology.orchestra.run.smithi046.stdout:584 trunc from 0x974b732 to 0x7d28ea2 2024-09-12T07:19:50.149 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:50.151+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:50.154 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:50.162 INFO:teuthology.orchestra.run.smithi046.stdout:587 write 0xa6a3ea3 thru 0xa6b3d35 (0xfe93 bytes) 2024-09-12T07:19:50.165 INFO:teuthology.orchestra.run.smithi046.stdout:588 writesame 0x1e21dcc thru 0x1e27732 data_size 0x3(0x5967 bytes) 2024-09-12T07:19:50.169 INFO:teuthology.orchestra.run.smithi046.stdout:591 punch from 0x14b17be to 0x14b1aad, (0x2ef bytes) 2024-09-12T07:19:50.169 INFO:teuthology.orchestra.run.smithi046.stdout:592 writesame 0x1ce7473 thru 0x1cf7289 data_size 0x1(0xfe17 bytes) 2024-09-12T07:19:50.201 INFO:teuthology.orchestra.run.smithi046.stdout:593 writesame 0xa2d1185 thru 0xa2dd244 data_size 0x40(0xc0c0 bytes) 2024-09-12T07:19:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:594 read 0x54a6044 thru 0x54b0502 (0xa4bf bytes) 2024-09-12T07:19:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:50.219 INFO:teuthology.orchestra.run.smithi046.stdout:596 trunc from 0xa6b3d36 to 0x7214f1 2024-09-12T07:19:50.246 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:50.247+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:50.250 INFO:teuthology.orchestra.run.smithi046.stdout:598 write 0x9113ba1 thru 0x9113c67 (0xc7 bytes) 2024-09-12T07:19:50.253 INFO:teuthology.orchestra.run.smithi046.stdout:600 punch from 0x18a73de to 0x18b3ad0, (0xc6f2 bytes) 2024-09-12T07:19:50.253 INFO:teuthology.orchestra.run.smithi046.stdout:601 read 0x69a933f thru 0x69b741e (0xe0e0 bytes) 2024-09-12T07:19:50.253 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:50.253 INFO:teuthology.orchestra.run.smithi046.stdout:603 punch from 0x3239e22 to 0x32483ff, (0xe5dd bytes) 2024-09-12T07:19:50.258 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:50.259 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:50.264 INFO:teuthology.orchestra.run.smithi046.stdout:605 writesame 0xde8490c thru 0xde8cd73 data_size 0x4c(0x8468 bytes) 2024-09-12T07:19:50.264 INFO:teuthology.orchestra.run.smithi046.stdout:607 write 0x3bbe41e thru 0x3bc48cf (0x64b2 bytes) 2024-09-12T07:19:50.265 INFO:teuthology.orchestra.run.smithi046.stdout:610 read 0x7388f8e thru 0x738992a (0x99d bytes) 2024-09-12T07:19:50.269 INFO:teuthology.orchestra.run.smithi046.stdout:611 write 0x7b658e0 thru 0x7b703da (0xaafb bytes) 2024-09-12T07:19:50.269 INFO:teuthology.orchestra.run.smithi046.stdout:613 read 0x7bc4aac thru 0x7bd48aa (0xfdff bytes) 2024-09-12T07:19:50.270 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:50.270 INFO:teuthology.orchestra.run.smithi046.stdout:615 read 0x8099080 thru 0x809c60f (0x3590 bytes) 2024-09-12T07:19:50.270 INFO:teuthology.orchestra.run.smithi046.stdout:617 write 0x56afcd7 thru 0x56b25f4 (0x291e bytes) 2024-09-12T07:19:50.271 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:19:50.283 INFO:teuthology.orchestra.run.smithi046.stdout:618 trunc from 0xde8cd74 to 0x30b9005 2024-09-12T07:19:50.297 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:50.297 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:50.297 INFO:teuthology.orchestra.run.smithi046.stdout:621 read 0xf56b07 thru 0xf66040 (0xf53a bytes) 2024-09-12T07:19:50.297 INFO:teuthology.orchestra.run.smithi046.stdout:624 read 0xa14f44 thru 0xa1c4c1 (0x757e bytes) 2024-09-12T07:19:50.307 INFO:teuthology.orchestra.run.smithi046.stdout:625 writesame 0x6df1842 thru 0x6dfc934 data_size 0x1(0xb0f3 bytes) 2024-09-12T07:19:50.317 INFO:teuthology.orchestra.run.smithi046.stdout:626 writesame 0xa3ce330 thru 0xa3d46bd data_size 0x1(0x638e bytes) 2024-09-12T07:19:50.317 INFO:teuthology.orchestra.run.smithi046.stdout:627 read 0x1a2beeb thru 0x1a3a5cc (0xe6e2 bytes) 2024-09-12T07:19:50.317 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:50.317 INFO:teuthology.orchestra.run.smithi046.stdout:630 punch from 0x10796a1 to 0x107ff43, (0x68a2 bytes) 2024-09-12T07:19:50.323 INFO:teuthology.orchestra.run.smithi046.stdout:631 write 0xb0e6848 thru 0xb0ec052 (0x580b bytes) 2024-09-12T07:19:50.326 INFO:teuthology.orchestra.run.smithi046.stdout:632 write 0x355fe8f thru 0x3565100 (0x5272 bytes) 2024-09-12T07:19:50.328 INFO:teuthology.orchestra.run.smithi046.stdout:633 read 0x2059ab2 thru 0x206813d (0xe68c bytes) 2024-09-12T07:19:50.328 INFO:teuthology.orchestra.run.smithi046.stdout:635 read 0x5e8c253 thru 0x5e91a87 (0x5835 bytes) 2024-09-12T07:19:50.329 INFO:teuthology.orchestra.run.smithi046.stdout:636 read 0xa4c4760 thru 0xa4c8eb8 (0x4759 bytes) 2024-09-12T07:19:50.329 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:50.329 INFO:teuthology.orchestra.run.smithi046.stdout:641 punch from 0x570384f to 0x5707adc, (0x428d bytes) 2024-09-12T07:19:50.335 INFO:teuthology.orchestra.run.smithi046.stdout:642 trunc from 0xb0ec053 to 0x5637ab8 2024-09-12T07:19:50.346 INFO:teuthology.orchestra.run.smithi046.stdout:643 read 0x305b12a thru 0x305b30c (0x1e3 bytes) 2024-09-12T07:19:50.346 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:50.348+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:50.353 INFO:teuthology.orchestra.run.smithi046.stdout:644 trunc from 0x5637ab8 to 0xbaf2d82 2024-09-12T07:19:50.358 INFO:teuthology.orchestra.run.smithi046.stdout:646 writesame 0xdf05fc thru 0xdf5742 data_size 0x1(0x5147 bytes) 2024-09-12T07:19:50.368 INFO:teuthology.orchestra.run.smithi046.stdout:647 read 0x12e5d5e thru 0x12f4932 (0xebd5 bytes) 2024-09-12T07:19:50.368 INFO:teuthology.orchestra.run.smithi046.stdout:648 write 0x37b4859 thru 0x37b7bb0 (0x3358 bytes) 2024-09-12T07:19:50.372 INFO:teuthology.orchestra.run.smithi046.stdout:649 write 0x1c019d8 thru 0x1c02994 (0xfbd bytes) 2024-09-12T07:19:50.378 INFO:teuthology.orchestra.run.smithi046.stdout:650 trunc from 0xbaf2d82 to 0x79a7c92 2024-09-12T07:19:50.381 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:50.381 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:50.388 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:50.390 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:19:50.402 INFO:teuthology.orchestra.run.smithi046.stdout:652 writesame 0xe7958a3 thru 0xe7a2f96 data_size 0x2(0xd6f4 bytes) 2024-09-12T07:19:50.402 INFO:teuthology.orchestra.run.smithi046.stdout:654 write 0x6217b81 thru 0x62257b6 (0xdc36 bytes) 2024-09-12T07:19:50.403 INFO:teuthology.orchestra.run.smithi046.stdout:655 read 0x60dd483 thru 0x60ebc38 (0xe7b6 bytes) 2024-09-12T07:19:50.417 INFO:teuthology.orchestra.run.smithi046.stdout:656 trunc from 0xe7a2f97 to 0x25288bb 2024-09-12T07:19:50.436 INFO:teuthology.orchestra.run.smithi046.stdout:658 writesame 0x6308b22 thru 0x6313970 data_size 0x1(0xae4f bytes) 2024-09-12T07:19:50.437 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:50.437 INFO:teuthology.orchestra.run.smithi046.stdout:660 read 0x1fa049f thru 0x1fa4a13 (0x4575 bytes) 2024-09-12T07:19:50.437 INFO:teuthology.orchestra.run.smithi046.stdout:662 read 0x503071 thru 0x50bace (0x8a5e bytes) 2024-09-12T07:19:50.438 INFO:teuthology.orchestra.run.smithi046.stdout:663 read 0x50687fd thru 0x506df60 (0x5764 bytes) 2024-09-12T07:19:50.447 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:50.448+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:50.451 INFO:teuthology.orchestra.run.smithi046.stdout:664 write 0xd139a7e thru 0xd141e41 (0x83c4 bytes) 2024-09-12T07:19:50.455 INFO:teuthology.orchestra.run.smithi046.stdout:666 writesame 0xbfe0a00 thru 0xbff014f data_size 0x18(0xf750 bytes) 2024-09-12T07:19:50.459 INFO:teuthology.orchestra.run.smithi046.stdout:667 write 0xbb5d43b thru 0xbb6c1e8 (0xedae bytes) 2024-09-12T07:19:50.461 INFO:teuthology.orchestra.run.smithi046.stdout:668 read 0x9b5a7b9 thru 0x9b69b39 (0xf381 bytes) 2024-09-12T07:19:50.462 INFO:teuthology.orchestra.run.smithi046.stdout:669 read 0x6aaf5cd thru 0x6ab881f (0x9253 bytes) 2024-09-12T07:19:50.462 INFO:teuthology.orchestra.run.smithi046.stdout:670 read 0x62a8b4c thru 0x62ab09d (0x2552 bytes) 2024-09-12T07:19:50.470 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:19:50.476 INFO:teuthology.orchestra.run.smithi046.stdout:672 trunc from 0xd141e42 to 0x1b4d7bb 2024-09-12T07:19:50.492 INFO:teuthology.orchestra.run.smithi046.stdout:673 punch from 0x152dbc3 to 0x1532358, (0x4795 bytes) 2024-09-12T07:19:50.492 INFO:teuthology.orchestra.run.smithi046.stdout:674 punch from 0xb6d28b to 0xb79a34, (0xc7a9 bytes) 2024-09-12T07:19:50.492 INFO:teuthology.orchestra.run.smithi046.stdout:675 clone 4 order 18 su 1024 sc 5 2024-09-12T07:19:50.541 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:50.542 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:50.548 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:50.549+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:50.550 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:19:50.649 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:50.651+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:50.696 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:50.696 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:50.705 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:19:50.750 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:50.751+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:50.811 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:50.811 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:50.820 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:19:50.865 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:50.866+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:50.928 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:50.928 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:50.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:19:50.977 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:50.979+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:51.052 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:51.054+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:51.083 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:51.083 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:51.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:19:51.153 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:51.155+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:51.198 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:51.199 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:51.207 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:19:51.254 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:51.256+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:51.328 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:51.328 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:51.337 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:19:51.355 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:51.357+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:51.445 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:51.446 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:51.454 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:19:51.455 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:51.457+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:51.545 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone3 from 0x1b4d7bb (overlap 0x7214f1) to 0x4fdb75 2024-09-12T07:19:51.556 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:51.557+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:51.577 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:51.577 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:51.587 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:19:51.600 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:19:51.600 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:19:51.601 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:19:51.601 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-12T07:19:51.624 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-12T07:19:51.634 INFO:teuthology.orchestra.run.smithi046.stdout:676 punch from 0xcf04bc to 0xcf5565, (0x50a9 bytes) 2024-09-12T07:19:51.655 INFO:teuthology.orchestra.run.smithi046.stdout:677 write 0xba1a204 thru 0xba24624 (0xa421 bytes) 2024-09-12T07:19:51.657 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:51.659+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:51.664 INFO:teuthology.orchestra.run.smithi046.stdout:678 read 0x56acfb4 thru 0x56b22a7 (0x52f4 bytes) 2024-09-12T07:19:51.664 INFO:teuthology.orchestra.run.smithi046.stdout:680 read 0x4be8da0 thru 0x4bee1c9 (0x542a bytes) 2024-09-12T07:19:51.664 INFO:teuthology.orchestra.run.smithi046.stdout:681 punch from 0xb696b95 to 0xb69e2ad, (0x7718 bytes) 2024-09-12T07:19:51.664 INFO:teuthology.orchestra.run.smithi046.stdout:682 punch from 0xb57a2b5 to 0xb589f2f, (0xfc7a bytes) 2024-09-12T07:19:51.664 INFO:teuthology.orchestra.run.smithi046.stdout:683 read 0x47d6a0f thru 0x47d8041 (0x1633 bytes) 2024-09-12T07:19:51.664 INFO:teuthology.orchestra.run.smithi046.stdout:684 read 0x9cbd57e thru 0x9cc3056 (0x5ad9 bytes) 2024-09-12T07:19:51.664 INFO:teuthology.orchestra.run.smithi046.stdout:685 punch from 0xa68b8c6 to 0xa698d69, (0xd4a3 bytes) 2024-09-12T07:19:51.664 INFO:teuthology.orchestra.run.smithi046.stdout:686 write 0x838f0dc thru 0x8398c37 (0x9b5c bytes) 2024-09-12T07:19:51.673 INFO:teuthology.orchestra.run.smithi046.stdout:687 read 0x56579df thru 0x565c2e6 (0x4908 bytes) 2024-09-12T07:19:51.673 INFO:teuthology.orchestra.run.smithi046.stdout:689 clone 5 order 24 su 131072 sc 4 2024-09-12T07:19:51.742 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:51.742 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:51.751 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:19:51.758 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:51.760+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:51.859 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:51.861+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:51.898 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:51.899 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:51.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:19:51.944 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-12T07:19:51.945+0000 7f0757424640 -1 mon.a@0(leader).osd e32 definitely_dead 0 2024-09-12T07:19:51.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:19:51.960 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:51.962+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:52.015 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:52.015 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:52.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:19:52.061 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:52.063+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:52.080 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:19:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:19:52.098 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:19:52.098 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:19:52.117 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:19:52.117 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:19:52.131 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:19:52.132 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:19:52.134 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:52.134 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:52.135 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:19:52.135 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:52.144 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:19:52.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:19:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:19:52.161 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:52.163+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:52.166 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:19:52.166 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:19:52.183 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:19:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:19:52.200 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:19:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:19:52.252 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:52.252 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:52.252 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:19:52.252 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:19:52.252 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:52.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:19:52.302 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:52.303+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:52.364 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:52.366+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:52.410 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:19:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:19:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:19:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.410 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4244.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:30.217471+0000", 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.193395348999999, 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014488159999999999, 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217471+0000", 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217471+0000", 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217472+0000", 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4200000000000002e-07 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217482+0000", 2024-09-12T07:19:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0067e-05 2024-09-12T07:19:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217494+0000", 2024-09-12T07:19:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2437e-05 2024-09-12T07:19:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.217522+0000", 2024-09-12T07:19:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7929999999999999e-05 2024-09-12T07:19:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.218022+0000", 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00049994099999999995 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.218103+0000", 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1038999999999994e-05 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.218872+0000", 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00076892799999999999 2024-09-12T07:19:52.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:30.218920+0000", 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8131999999999999e-05 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:12 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.467367+0000", 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.9434996949999999, 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027275540000000001, 2024-09-12T07:19:52.415 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467367+0000", 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467367+0000", 2024-09-12T07:19:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467368+0000", 2024-09-12T07:19:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1519999999999999e-06 2024-09-12T07:19:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467372+0000", 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1980000000000003e-06 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467380+0000", 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.515e-06 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467387+0000", 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0570000000000004e-06 2024-09-12T07:19:52.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467437+0000", 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9738999999999999e-05 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467447+0000", 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0227999999999999e-05 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.468920+0000", 2024-09-12T07:19:52.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014732059999999999 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469023+0000", 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103087 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469168+0000", 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014458099999999999 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469640+0000", 2024-09-12T07:19:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047246499999999998 2024-09-12T07:19:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470041+0000", 2024-09-12T07:19:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040072399999999998 2024-09-12T07:19:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470072+0000", 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1080000000000001e-05 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470095+0000", 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2521999999999999e-05 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.422 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:492 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [watch unwatch cookie 140539785655440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.555907+0000", 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.8549593020000001, 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029325409999999999, 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 492 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555907+0000", 2024-09-12T07:19:52.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555907+0000", 2024-09-12T07:19:52.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555908+0000", 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4499999999999997e-07 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555912+0000", 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6150000000000001e-06 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555923+0000", 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0832000000000001e-05 2024-09-12T07:19:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555930+0000", 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7279999999999995e-06 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556075+0000", 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014423300000000001 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556146+0000", 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1710000000000003e-05 2024-09-12T07:19:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556235+0000", 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9161999999999998e-05 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556559+0000", 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000323407 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558730+0000", 2024-09-12T07:19:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021707190000000002 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558761+0000", 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1504000000000002e-05 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558840+0000", 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8986000000000006e-05 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:493 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.556112+0000", 2024-09-12T07:19:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.854754582, 2024-09-12T07:19:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029569990000000001, 2024-09-12T07:19:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.429 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 493 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556112+0000", 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556112+0000", 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556113+0000", 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6799999999999999e-07 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556117+0000", 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2239999999999997e-06 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556125+0000", 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9799999999999998e-06 2024-09-12T07:19:52.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556129+0000", 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1559999999999997e-06 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556391+0000", 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026192499999999998 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for rw locks", 2024-09-12T07:19:52.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556429+0000", 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7280000000000002e-05 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558887+0000", 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024582580000000001 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558980+0000", 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3177999999999995e-05 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.559069+0000", 2024-09-12T07:19:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9229999999999998e-05 2024-09-12T07:19:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:610 3.9 3:911399f5:::rbd_data.116011b0bca6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.685417+0000", 2024-09-12T07:19:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.7254495670000001, 2024-09-12T07:19:52.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001501474, 2024-09-12T07:19:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 610 2024-09-12T07:19:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685417+0000", 2024-09-12T07:19:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685417+0000", 2024-09-12T07:19:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685419+0000", 2024-09-12T07:19:52.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.739e-06 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685427+0000", 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8890000000000002e-06 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685443+0000", 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6586999999999999e-05 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685450+0000", 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0729999999999999e-06 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685511+0000", 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0596000000000001e-05 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685613+0000", 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010168 2024-09-12T07:19:52.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685756+0000", 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000143237 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686751+0000", 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099451300000000004 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:52.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686802+0000", 2024-09-12T07:19:52.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0998000000000002e-05 2024-09-12T07:19:52.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686873+0000", 2024-09-12T07:19:52.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1029e-05 2024-09-12T07:19:52.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686919+0000", 2024-09-12T07:19:52.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6133e-05 2024-09-12T07:19:52.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:956 3.9 3:96f4f606:::rbd_data.116011b0bca6.0000000000000027:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.206153+0000", 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.204713527, 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016258261999999999, 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 956 2024-09-12T07:19:52.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.442 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206153+0000", 2024-09-12T07:19:52.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206153+0000", 2024-09-12T07:19:52.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206155+0000", 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0499999999999999e-06 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206163+0000", 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6699999999999994e-06 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206181+0000", 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7645000000000001e-05 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206188+0000", 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0829999999999998e-06 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206239+0000", 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0973000000000001e-05 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206346+0000", 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010729400000000001 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.209070+0000", 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027238340000000001 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222329+0000", 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013259655 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:52.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222341+0000", 2024-09-12T07:19:52.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.119e-05 2024-09-12T07:19:52.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222378+0000", 2024-09-12T07:19:52.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7008000000000002e-05 2024-09-12T07:19:52.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222411+0000", 2024-09-12T07:19:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3859999999999998e-05 2024-09-12T07:19:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:956 3.9 e28/24 ondisk, result = 0)", 2024-09-12T07:19:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.217233+0000", 2024-09-12T07:19:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.1936338790000001, 2024-09-12T07:19:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051019610000000003, 2024-09-12T07:19:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217233+0000", 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217233+0000", 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217234+0000", 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.579e-06 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217243+0000", 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3310000000000004e-06 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217258+0000", 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4715000000000001e-05 2024-09-12T07:19:52.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217267+0000", 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1439999999999992e-06 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222316+0000", 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050498039999999998 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222325+0000", 2024-09-12T07:19:52.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5480000000000002e-06 2024-09-12T07:19:52.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222335+0000", 2024-09-12T07:19:52.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8400000000000007e-06 2024-09-12T07:19:52.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.451 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.451 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:960 3.9 3:911399f5:::rbd_data.116011b0bca6.0000000000000007:head [stat,write 228792~62787] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.236487+0000", 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.1743798529999996, 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016742370000000001, 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 960 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236487+0000", 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236487+0000", 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236489+0000", 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.686e-06 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236510+0000", 2024-09-12T07:19:52.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1060000000000002e-05 2024-09-12T07:19:52.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236522+0000", 2024-09-12T07:19:52.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.256e-05 2024-09-12T07:19:52.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236528+0000", 2024-09-12T07:19:52.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9150000000000001e-06 2024-09-12T07:19:52.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236588+0000", 2024-09-12T07:19:52.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9398999999999998e-05 2024-09-12T07:19:52.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236692+0000", 2024-09-12T07:19:52.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104589 2024-09-12T07:19:52.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:52.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236857+0000", 2024-09-12T07:19:52.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016502699999999999 2024-09-12T07:19:52.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:52.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.237713+0000", 2024-09-12T07:19:52.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00085576399999999996 2024-09-12T07:19:52.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:52.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.238083+0000", 2024-09-12T07:19:52.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037034999999999998 2024-09-12T07:19:52.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.238115+0000", 2024-09-12T07:19:52.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2211e-05 2024-09-12T07:19:52.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.238161+0000", 2024-09-12T07:19:52.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5676000000000001e-05 2024-09-12T07:19:52.461 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.461 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.461 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.462 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:984 3.9 3:96f4f606:::rbd_data.116011b0bca6.0000000000000027:head [write 1013176~35400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:52.462 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.269836+0000", 2024-09-12T07:19:52.462 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.1410306280000002, 2024-09-12T07:19:52.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020110530000000001, 2024-09-12T07:19:52.462 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.463 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 984 2024-09-12T07:19:52.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.464 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269836+0000", 2024-09-12T07:19:52.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269836+0000", 2024-09-12T07:19:52.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269838+0000", 2024-09-12T07:19:52.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7770000000000001e-06 2024-09-12T07:19:52.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269857+0000", 2024-09-12T07:19:52.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9029999999999999e-05 2024-09-12T07:19:52.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269874+0000", 2024-09-12T07:19:52.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7359000000000001e-05 2024-09-12T07:19:52.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269882+0000", 2024-09-12T07:19:52.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:19:52.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269938+0000", 2024-09-12T07:19:52.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5801999999999999e-05 2024-09-12T07:19:52.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.270002+0000", 2024-09-12T07:19:52.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4448000000000004e-05 2024-09-12T07:19:52.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:52.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.270086+0000", 2024-09-12T07:19:52.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3919999999999996e-05 2024-09-12T07:19:52.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:52.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.270707+0000", 2024-09-12T07:19:52.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00062067299999999997 2024-09-12T07:19:52.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:52.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.271735+0000", 2024-09-12T07:19:52.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010287720000000001 2024-09-12T07:19:52.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.271796+0000", 2024-09-12T07:19:52.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0791000000000001e-05 2024-09-12T07:19:52.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.271847+0000", 2024-09-12T07:19:52.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1070999999999999e-05 2024-09-12T07:19:52.475 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.475 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.475 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.476 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1111 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:52.476 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.493764+0000", 2024-09-12T07:19:52.476 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9171025579999998, 2024-09-12T07:19:52.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020530140000000001, 2024-09-12T07:19:52.477 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.477 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:52.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1111 2024-09-12T07:19:52.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.478 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493764+0000", 2024-09-12T07:19:52.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493764+0000", 2024-09-12T07:19:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493766+0000", 2024-09-12T07:19:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6440000000000001e-06 2024-09-12T07:19:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493774+0000", 2024-09-12T07:19:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0749999999999998e-06 2024-09-12T07:19:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493791+0000", 2024-09-12T07:19:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6799e-05 2024-09-12T07:19:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493801+0000", 2024-09-12T07:19:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9939999999999995e-06 2024-09-12T07:19:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493859+0000", 2024-09-12T07:19:52.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8269000000000001e-05 2024-09-12T07:19:52.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:52.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493864+0000", 2024-09-12T07:19:52.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4970000000000001e-06 2024-09-12T07:19:52.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.495607+0000", 2024-09-12T07:19:52.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001742586 2024-09-12T07:19:52.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.495670+0000", 2024-09-12T07:19:52.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3077000000000006e-05 2024-09-12T07:19:52.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.495817+0000", 2024-09-12T07:19:52.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014707300000000001 2024-09-12T07:19:52.487 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.487 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.488 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.488 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1345 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7340032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:52.488 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.755762+0000", 2024-09-12T07:19:52.489 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.655104734, 2024-09-12T07:19:52.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0066881680000000004, 2024-09-12T07:19:52.489 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.489 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.489 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.489 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.490 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1345 2024-09-12T07:19:52.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.490 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755762+0000", 2024-09-12T07:19:52.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755762+0000", 2024-09-12T07:19:52.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755764+0000", 2024-09-12T07:19:52.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2199999999999999e-06 2024-09-12T07:19:52.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756112+0000", 2024-09-12T07:19:52.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000347287 2024-09-12T07:19:52.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756136+0000", 2024-09-12T07:19:52.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4488e-05 2024-09-12T07:19:52.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756147+0000", 2024-09-12T07:19:52.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0857e-05 2024-09-12T07:19:52.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756221+0000", 2024-09-12T07:19:52.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4200000000000001e-05 2024-09-12T07:19:52.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756351+0000", 2024-09-12T07:19:52.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012945 2024-09-12T07:19:52.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:52.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756621+0000", 2024-09-12T07:19:52.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027040400000000002 2024-09-12T07:19:52.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:52.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759352+0000", 2024-09-12T07:19:52.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027308290000000002 2024-09-12T07:19:52.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:52.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.762330+0000", 2024-09-12T07:19:52.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029781170000000002 2024-09-12T07:19:52.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.762400+0000", 2024-09-12T07:19:52.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0220000000000002e-05 2024-09-12T07:19:52.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.762450+0000", 2024-09-12T07:19:52.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0096000000000003e-05 2024-09-12T07:19:52.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.498 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.499 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1561 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:head [stat,truncate 1280616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:52.499 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.997537+0000", 2024-09-12T07:19:52.499 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.4133297169999999, 2024-09-12T07:19:52.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002027367, 2024-09-12T07:19:52.499 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.499 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1561 2024-09-12T07:19:52.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.500 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997537+0000", 2024-09-12T07:19:52.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997537+0000", 2024-09-12T07:19:52.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997539+0000", 2024-09-12T07:19:52.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.215e-06 2024-09-12T07:19:52.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997550+0000", 2024-09-12T07:19:52.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0763e-05 2024-09-12T07:19:52.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997572+0000", 2024-09-12T07:19:52.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1647000000000001e-05 2024-09-12T07:19:52.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997581+0000", 2024-09-12T07:19:52.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8020000000000003e-06 2024-09-12T07:19:52.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997650+0000", 2024-09-12T07:19:52.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8845999999999998e-05 2024-09-12T07:19:52.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997735+0000", 2024-09-12T07:19:52.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4915999999999998e-05 2024-09-12T07:19:52.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:52.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997872+0000", 2024-09-12T07:19:52.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000136365 2024-09-12T07:19:52.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:52.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.998859+0000", 2024-09-12T07:19:52.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00098759400000000006 2024-09-12T07:19:52.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:52.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999454+0000", 2024-09-12T07:19:52.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00059463300000000001 2024-09-12T07:19:52.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999523+0000", 2024-09-12T07:19:52.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9289999999999996e-05 2024-09-12T07:19:52.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999564+0000", 2024-09-12T07:19:52.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1295999999999999e-05 2024-09-12T07:19:52.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.507 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1701 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:head [stat,truncate 524288] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.564930+0000", 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.8459367279999999, 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045744499999999999, 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1701 2024-09-12T07:19:52.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.564930+0000", 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.564930+0000", 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.564932+0000", 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9970000000000001e-06 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.564962+0000", 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0323000000000001e-05 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.564985+0000", 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2762999999999999e-05 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.564995+0000", 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8810000000000008e-06 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.565066+0000", 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1223999999999994e-05 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.565169+0000", 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010238699999999999 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.565386+0000", 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021692099999999999 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.566812+0000", 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001426229 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.569440+0000", 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026286249999999999 2024-09-12T07:19:52.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.569475+0000", 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4482e-05 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.569504+0000", 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9618e-05 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.513 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1922 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:5 [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.803831+0000", 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.607035964, 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014842900000000001, 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1922 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803831+0000", 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803831+0000", 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803831+0000", 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.39e-07 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803833+0000", 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9860000000000001e-06 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803839+0000", 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6799999999999998e-06 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803843+0000", 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5900000000000001e-06 2024-09-12T07:19:52.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805124+0000", 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001280158 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805211+0000", 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7288999999999997e-05 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805315+0000", 2024-09-12T07:19:52.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010414800000000001 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1923 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:5 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.803850+0000", 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.6070166459999999, 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016816719999999999, 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1923 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803850+0000", 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803850+0000", 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803851+0000", 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.01e-07 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803852+0000", 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9319999999999999e-06 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803859+0000", 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:19:52.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803863+0000", 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3669999999999999e-06 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805353+0000", 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014896760000000001 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805428+0000", 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4868000000000002e-05 2024-09-12T07:19:52.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805532+0000", 2024-09-12T07:19:52.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104248 2024-09-12T07:19:52.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.525 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.525 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1924 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:5 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.803908+0000", 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.6069590639999998, 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0018887890000000001, 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1924 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803908+0000", 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803908+0000", 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803908+0000", 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9999999999999996e-07 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803913+0000", 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3760000000000001e-06 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803924+0000", 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1092e-05 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803932+0000", 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6199999999999999e-06 2024-09-12T07:19:52.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805574+0000", 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016424040000000001 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805673+0000", 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8936000000000003e-05 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805796+0000", 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012356100000000001 2024-09-12T07:19:52.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1958 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:5 [sparse-read 5242880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.811232+0000", 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.5996350669999999, 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014818979999999999, 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1958 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811232+0000", 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811232+0000", 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811232+0000", 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3700000000000001e-07 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811234+0000", 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2819999999999999e-06 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811241+0000", 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6959999999999999e-06 2024-09-12T07:19:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811250+0000", 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6370000000000005e-06 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.812521+0000", 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001270897 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.812605+0000", 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4518000000000006e-05 2024-09-12T07:19:52.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.812714+0000", 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010853099999999999 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1959 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:5 [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.811258+0000", 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.5996086410000001, 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016934879999999999, 2024-09-12T07:19:52.534 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1959 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811258+0000", 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811258+0000", 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811259+0000", 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8100000000000003e-07 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811261+0000", 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8480000000000001e-06 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811268+0000", 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.024e-06 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811274+0000", 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3170000000000001e-06 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.812750+0000", 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014765710000000001 2024-09-12T07:19:52.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.812836+0000", 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5484999999999999e-05 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.812952+0000", 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011576199999999999 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1960 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:5 [sparse-read 7340032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.811282+0000", 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.5995851779999999, 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019447640000000001, 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1960 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811282+0000", 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811282+0000", 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811282+0000", 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4700000000000002e-07 2024-09-12T07:19:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811292+0000", 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8490000000000001e-06 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811299+0000", 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8399999999999997e-06 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811304+0000", 2024-09-12T07:19:52.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1499999999999998e-06 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.813019+0000", 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017152549999999999 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.813112+0000", 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2807999999999996e-05 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.813226+0000", 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011441499999999999 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1995 3.9 3:951701e1:::rbd_data.1160947fa40f.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.844656+0000", 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.5662111749999998, 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028069369999999998, 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.543 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1995 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844656+0000", 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844656+0000", 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844657+0000", 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.722e-06 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844664+0000", 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8059999999999997e-06 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844680+0000", 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6181999999999999e-05 2024-09-12T07:19:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844688+0000", 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2690000000000002e-06 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844752+0000", 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4471e-05 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844847+0000", 2024-09-12T07:19:52.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4857000000000003e-05 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844971+0000", 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012368000000000001 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.845521+0000", 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00055040299999999998 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.847359+0000", 2024-09-12T07:19:52.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0018378330000000001 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.847416+0000", 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7661999999999998e-05 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.847463+0000", 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6051999999999999e-05 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.549 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.549 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:52.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:19:52.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:19:52.551 INFO:teuthology.orchestra.run.smithi046.stderr:marked out osd.3. 2024-09-12T07:19:52.552 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:52.551+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:52.651 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:52.653+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:52.701 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:52.701 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:52.701 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:19:52.702 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:52.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:19:52.752 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:52.753+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:52.853 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:52.854+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:52.858 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:52.858 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:52.858 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:19:52.859 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:19:52.859 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:52.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:19:52.954 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:52.955+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:52.977 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:52.977 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:19:52.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:19:52.977 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:19:52.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.977 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:613 3.7 3:e175feac:::rbd_data.116011b0bca6.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:52.977 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.686095+0000", 2024-09-12T07:19:52.978 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.2914961419999997, 2024-09-12T07:19:52.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054618599999999998, 2024-09-12T07:19:52.978 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.978 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.978 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.978 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 613 2024-09-12T07:19:52.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.978 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686095+0000", 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686095+0000", 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686097+0000", 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0699999999999999e-06 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686102+0000", 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6570000000000002e-06 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686114+0000", 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1637999999999999e-05 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686123+0000", 2024-09-12T07:19:52.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6789999999999994e-06 2024-09-12T07:19:52.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686187+0000", 2024-09-12T07:19:52.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4734999999999996e-05 2024-09-12T07:19:52.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686337+0000", 2024-09-12T07:19:52.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000149383 2024-09-12T07:19:52.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:52.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686544+0000", 2024-09-12T07:19:52.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020754000000000001 2024-09-12T07:19:52.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:52.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687923+0000", 2024-09-12T07:19:52.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013792850000000001 2024-09-12T07:19:52.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:52.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691480+0000", 2024-09-12T07:19:52.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035560660000000001 2024-09-12T07:19:52.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691514+0000", 2024-09-12T07:19:52.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3979000000000001e-05 2024-09-12T07:19:52.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691557+0000", 2024-09-12T07:19:52.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3828000000000001e-05 2024-09-12T07:19:52.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.987 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.988 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:617 3.2 3:4522120e:::rbd_data.116011b0bca6.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:52.988 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.687024+0000", 2024-09-12T07:19:52.988 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.2905673059999998, 2024-09-12T07:19:52.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047272190000000004, 2024-09-12T07:19:52.989 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 617 2024-09-12T07:19:52.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.990 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687024+0000", 2024-09-12T07:19:52.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687024+0000", 2024-09-12T07:19:52.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687025+0000", 2024-09-12T07:19:52.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6000000000000002e-07 2024-09-12T07:19:52.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687055+0000", 2024-09-12T07:19:52.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.029e-05 2024-09-12T07:19:52.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687065+0000", 2024-09-12T07:19:52.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6749999999999997e-06 2024-09-12T07:19:52.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687072+0000", 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.657e-06 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687131+0000", 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9157000000000003e-05 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687263+0000", 2024-09-12T07:19:52.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013171799999999999 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687458+0000", 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019529899999999999 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.688377+0000", 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00091874700000000003 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691642+0000", 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0032655290000000001 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691706+0000", 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3548999999999998e-05 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691752+0000", 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5738000000000002e-05 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.996 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:894 3.2 3:468e8e09:::rbd_data.116011b0bca6.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.8746796979999996, 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015762987999999999, 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:52.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 894 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:52.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102913+0000", 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004e-06 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102918+0000", 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6260000000000003e-06 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:52.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102929+0000", 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1382e-05 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102934+0000", 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102988+0000", 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3683000000000001e-05 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.103072+0000", 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4135000000000007e-05 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.105687+0000", 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026149580000000001 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118570+0000", 2024-09-12T07:19:53.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012883034 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118585+0000", 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4797000000000001e-05 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118622+0000", 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7163000000000003e-05 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118675+0000", 2024-09-12T07:19:53.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3236000000000003e-05 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:894 3.2 e28/24 ondisk, result = 0)", 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.112622+0000", 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.8649701199999997, 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059558010000000002, 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112622+0000", 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112622+0000", 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112623+0000", 2024-09-12T07:19:53.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9800000000000002e-07 2024-09-12T07:19:53.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112627+0000", 2024-09-12T07:19:53.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0720000000000001e-06 2024-09-12T07:19:53.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112640+0000", 2024-09-12T07:19:53.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3131e-05 2024-09-12T07:19:53.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112649+0000", 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8710000000000003e-06 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118557+0000", 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059084740000000004 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118565+0000", 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5270000000000001e-06 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118577+0000", 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2728000000000001e-05 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:912 3.2 3:468e8e09:::rbd_data.116011b0bca6.000000000000003e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.143632+0000", 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.833959417, 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051497230000000001, 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.007 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 912 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143632+0000", 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143632+0000", 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143634+0000", 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0040000000000002e-06 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143641+0000", 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1199999999999996e-06 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143660+0000", 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8408999999999999e-05 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143670+0000", 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0312999999999999e-05 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143747+0000", 2024-09-12T07:19:53.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7399999999999998e-05 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143855+0000", 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010739999999999999 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.144004+0000", 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014922500000000001 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:53.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.144699+0000", 2024-09-12T07:19:53.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00069510800000000005 2024-09-12T07:19:53.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:53.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.148690+0000", 2024-09-12T07:19:53.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039904490000000001 2024-09-12T07:19:53.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:53.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.148741+0000", 2024-09-12T07:19:53.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1328000000000001e-05 2024-09-12T07:19:53.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.148782+0000", 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:993 3.7 3:e61faebc:::rbd_data.116011b0bca6.000000000000004c:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.295319+0000", 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.6822728099999997, 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079320569999999993, 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 993 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295319+0000", 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295319+0000", 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295321+0000", 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7e-06 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295328+0000", 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2729999999999997e-06 2024-09-12T07:19:53.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295344+0000", 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6033e-05 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295353+0000", 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2699999999999993e-06 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295409+0000", 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5507000000000002e-05 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295521+0000", 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112719 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.296644+0000", 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001123045 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303082+0000", 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064377510000000002 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303131+0000", 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9295999999999997e-05 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303195+0000", 2024-09-12T07:19:53.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4022999999999993e-05 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303251+0000", 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5439999999999998e-05 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1031 3.7 3:e61faebc:::rbd_data.116011b0bca6.000000000000004c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.377551+0000", 2024-09-12T07:19:53.019 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.6000406160000002, 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050361379999999999, 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1031 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377551+0000", 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377551+0000", 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377554+0000", 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7970000000000001e-06 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377565+0000", 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1612e-05 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377589+0000", 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3955999999999999e-05 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377603+0000", 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3540000000000001e-05 2024-09-12T07:19:53.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377698+0000", 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5172000000000003e-05 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377793+0000", 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5137999999999996e-05 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:53.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377920+0000", 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126518 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378626+0000", 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00070584200000000004 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.382513+0000", 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038871539999999999 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.382542+0000", 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9583000000000001e-05 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.382587+0000", 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4826000000000001e-05 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.025 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1080 3.b 3:d0645053:::rbd_object_map.116011b0bca6.0000000000000004:head [writefull 0~53] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.470376+0000", 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.507215618, 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029172759999999999, 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1080 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470376+0000", 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470376+0000", 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470379+0000", 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.571e-06 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470388+0000", 2024-09-12T07:19:53.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1279999999999997e-06 2024-09-12T07:19:53.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470405+0000", 2024-09-12T07:19:53.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7229000000000001e-05 2024-09-12T07:19:53.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470416+0000", 2024-09-12T07:19:53.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0781000000000001e-05 2024-09-12T07:19:53.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470489+0000", 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3366000000000002e-05 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470497+0000", 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0180000000000001e-06 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.472041+0000", 2024-09-12T07:19:53.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015442780000000001 2024-09-12T07:19:53.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.472118+0000", 2024-09-12T07:19:53.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6599000000000003e-05 2024-09-12T07:19:53.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:53.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.472198+0000", 2024-09-12T07:19:53.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9742999999999999e-05 2024-09-12T07:19:53.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.472726+0000", 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052847500000000004 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.473232+0000", 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00050537100000000001 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.473262+0000", 2024-09-12T07:19:53.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0827999999999997e-05 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.473293+0000", 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0889000000000002e-05 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1143 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.514700+0000", 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.4628911650000003, 2024-09-12T07:19:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029554020000000002, 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1143 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514700+0000", 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514700+0000", 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514701+0000", 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2900000000000003e-07 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514705+0000", 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8260000000000003e-06 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514713+0000", 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3440000000000001e-06 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514717+0000", 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9500000000000003e-06 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514757+0000", 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9672999999999998e-05 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514799+0000", 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1978999999999999e-05 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514927+0000", 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012807000000000001 2024-09-12T07:19:53.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.515330+0000", 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040297199999999998 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.517605+0000", 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022750610000000001 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:53.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.517634+0000", 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.853e-05 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.517656+0000", 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2268000000000001e-05 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1338 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.748194+0000", 2024-09-12T07:19:53.038 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.2293973679999999, 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038514669999999999, 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1338 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748194+0000", 2024-09-12T07:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748194+0000", 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748196+0000", 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4759999999999999e-06 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748647+0000", 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00045091599999999999 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748661+0000", 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4534e-05 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748668+0000", 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9009999999999997e-06 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748723+0000", 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4468000000000001e-05 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748830+0000", 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010779 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748996+0000", 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165162 2024-09-12T07:19:53.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.751387+0000", 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002391706 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.751907+0000", 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051961499999999996 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.751981+0000", 2024-09-12T07:19:53.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4319000000000005e-05 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.752046+0000", 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4579999999999995e-05 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1343 3.7 3:e78f2392:::rbd_data.116075042ea1.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7340032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.755402+0000", 2024-09-12T07:19:53.044 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.2221893809999997, 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0040308560000000002, 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1343 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755402+0000", 2024-09-12T07:19:53.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755402+0000", 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755405+0000", 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.937e-06 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755456+0000", 2024-09-12T07:19:53.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0642000000000001e-05 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755479+0000", 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3527e-05 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755492+0000", 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3132999999999999e-05 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755569+0000", 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6323999999999996e-05 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755722+0000", 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000153394 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755924+0000", 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020157500000000001 2024-09-12T07:19:53.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:53.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.757983+0000", 2024-09-12T07:19:53.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00205894 2024-09-12T07:19:53.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:53.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759317+0000", 2024-09-12T07:19:53.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013344229999999999 2024-09-12T07:19:53.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759382+0000", 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4969999999999996e-05 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759433+0000", 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0991e-05 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.050 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1344 3.18 3:1a9d4b56:::rbd_data.116075042ea1.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7340032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.755621+0000", 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.221970604, 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0040014960000000002, 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1344 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755621+0000", 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755621+0000", 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755622+0000", 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2600000000000001e-07 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755847+0000", 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022498 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755858+0000", 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1426e-05 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755866+0000", 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6559999999999992e-06 2024-09-12T07:19:53.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755973+0000", 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010666300000000001 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756091+0000", 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011859800000000001 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:53.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756678+0000", 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00058683100000000001 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759291+0000", 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002612582 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759551+0000", 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026036000000000001 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759585+0000", 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3516999999999998e-05 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759623+0000", 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7957000000000003e-05 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:1341 3.19 e29/24 3:98c9a0e4:::rbd_data.116075042ea1.0000000000000004:head v 29'99, mlcod=28'97)", 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.755920+0000", 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.2216712259999998, 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031212290000000001, 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755920+0000", 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755920+0000", 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755921+0000", 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6699999999999999e-07 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755950+0000", 2024-09-12T07:19:53.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9479999999999999e-05 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755961+0000", 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0352e-05 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755968+0000", 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0890000000000002e-06 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756047+0000", 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9684000000000005e-05 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756069+0000", 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1795999999999999e-05 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.758791+0000", 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002721472 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759042+0000", 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00025088900000000002 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1587 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.471457+0000", 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5061344920000002, 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031074520000000001, 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1587 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471457+0000", 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471457+0000", 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471459+0000", 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.728e-06 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471470+0000", 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1061e-05 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471487+0000", 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7333000000000002e-05 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471496+0000", 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7430000000000008e-06 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471561+0000", 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4633000000000003e-05 2024-09-12T07:19:53.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471567+0000", 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0630000000000002e-06 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.473095+0000", 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001528092 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.473152+0000", 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7540999999999997e-05 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.473365+0000", 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021284599999999999 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.473824+0000", 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00045857500000000002 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.474486+0000", 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00066262099999999996 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.474520+0000", 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3392999999999997e-05 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.474565+0000", 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4823000000000001e-05 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.067 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2034 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:5 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.849522+0000", 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.1280698920000001, 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028828289999999999, 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.068 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2034 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849522+0000", 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849522+0000", 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849522+0000", 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0699999999999998e-07 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849524+0000", 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0650000000000001e-06 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849530+0000", 2024-09-12T07:19:53.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2609999999999999e-06 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849535+0000", 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7949999999999998e-06 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852233+0000", 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026980350000000001 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852316+0000", 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2566999999999997e-05 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852405+0000", 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8698999999999999e-05 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2035 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:5 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:53.072 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.849543+0000", 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.1280491239999999, 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00306924, 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2035 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849543+0000", 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849543+0000", 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849543+0000", 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849545+0000", 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1270000000000001e-06 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849552+0000", 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6270000000000002e-06 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849557+0000", 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8749999999999999e-06 2024-09-12T07:19:53.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852435+0000", 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028788580000000002 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852515+0000", 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9939999999999997e-05 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852612+0000", 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6389000000000002e-05 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2036 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:5 [sparse-read 4718592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.849575+0000", 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.1280170730000001, 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00318892, 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2036 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849575+0000", 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849575+0000", 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849575+0000", 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849578+0000", 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8890000000000002e-06 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849586+0000", 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7149999999999998e-06 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849591+0000", 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6690000000000003e-06 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852640+0000", 2024-09-12T07:19:53.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030482 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852689+0000", 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9181000000000002e-05 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852763+0000", 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4756000000000001e-05 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2037 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:5 [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.849600+0000", 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.1279920749999999, 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003393662, 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2037 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.082 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849600+0000", 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849600+0000", 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849600+0000", 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9499999999999998e-07 2024-09-12T07:19:53.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849602+0000", 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849609+0000", 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.533e-06 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849617+0000", 2024-09-12T07:19:53.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6400000000000003e-06 2024-09-12T07:19:53.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852795+0000", 2024-09-12T07:19:53.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031781550000000002 2024-09-12T07:19:53.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852877+0000", 2024-09-12T07:19:53.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1249999999999996e-05 2024-09-12T07:19:53.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852993+0000", 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116663 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2038 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:5 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.849626+0000", 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.1279655910000002, 2024-09-12T07:19:53.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003577233, 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2038 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849626+0000", 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849626+0000", 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849626+0000", 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3599999999999999e-07 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849629+0000", 2024-09-12T07:19:53.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3309999999999998e-06 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849635+0000", 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2790000000000004e-06 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849640+0000", 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230000000000001e-06 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.853024+0000", 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0033838850000000001 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.853106+0000", 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2068000000000001e-05 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.853203+0000", 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7311000000000004e-05 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2039 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:5 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.849648+0000", 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.1279436559999998, 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038036390000000002, 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2039 2024-09-12T07:19:53.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849648+0000", 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849648+0000", 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849648+0000", 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.03e-07 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849653+0000", 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1420000000000003e-06 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849659+0000", 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9489999999999999e-06 2024-09-12T07:19:53.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849664+0000", 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0019999999999997e-06 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.853242+0000", 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035776839999999998 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.853336+0000", 2024-09-12T07:19:53.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4136999999999997e-05 2024-09-12T07:19:53.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:53.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:53.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:53.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.853452+0000", 2024-09-12T07:19:53.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115322 2024-09-12T07:19:53.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.095 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:53.095 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:53.095 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:53.097 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:19:53.097 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:53.098+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:53.198 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:53.200+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:53.217 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:19:53.217 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:53.226 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:19:53.299 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:53.300+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:53.334 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:53.334 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:53.334 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:19:53.334 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:19:53.334 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:53.343 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:19:53.400 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:53.402+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:19:53.494 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:19:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:19:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:19:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:592 3.f e28/24 3:f4a2090e:::rbd_object_map.116011b0bca6:head v 28'17, mlcod=28'14)", 2024-09-12T07:19:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.678290+0000", 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.8163203499999998, 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0067778999999999999, 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678290+0000", 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678290+0000", 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678291+0000", 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.68e-07 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678313+0000", 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1888000000000001e-05 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678348+0000", 2024-09-12T07:19:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5063e-05 2024-09-12T07:19:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678373+0000", 2024-09-12T07:19:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4973e-05 2024-09-12T07:19:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678448+0000", 2024-09-12T07:19:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5743999999999999e-05 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678482+0000", 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3318999999999997e-05 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685027+0000", 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0065449169999999999 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685068+0000", 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1328000000000001e-05 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:594 3.f e28/24 3:f4a2090e:::rbd_object_map.116011b0bca6:head v 28'18, mlcod=28'14)", 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.678916+0000", 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.8156942029999996, 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006184691, 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678916+0000", 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678916+0000", 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678917+0000", 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8500000000000002e-07 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678921+0000", 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9330000000000003e-06 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678929+0000", 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9980000000000003e-06 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678933+0000", 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4229999999999998e-06 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678984+0000", 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0831000000000002e-05 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679007+0000", 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3430000000000001e-05 2024-09-12T07:19:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685070+0000", 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00606238 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685101+0000", 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1211000000000003e-05 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:600 3.f e28/24 3:f4a2090e:::rbd_object_map.116011b0bca6:head v 28'19, mlcod=28'14)", 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.679546+0000", 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.8150642359999996, 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059521690000000002, 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679546+0000", 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679546+0000", 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679547+0000", 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9200000000000001e-07 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679552+0000", 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6239999999999999e-06 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679564+0000", 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1704e-05 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679572+0000", 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7170000000000008e-06 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679626+0000", 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3841000000000002e-05 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679639+0000", 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3806e-05 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685471+0000", 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058312909999999997 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685498+0000", 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7594e-05 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.507 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:601 3.f e28/24 3:f4a2090e:::rbd_object_map.116011b0bca6:head v 28'20, mlcod=28'14)", 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.680100+0000", 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.8145106359999996, 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054139380000000001, 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680100+0000", 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680100+0000", 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680100+0000", 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9599999999999999e-07 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680114+0000", 2024-09-12T07:19:53.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3614999999999999e-05 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680122+0000", 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1210000000000007e-06 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680126+0000", 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4469999999999999e-06 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680178+0000", 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1515999999999999e-05 2024-09-12T07:19:53.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680201+0000", 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.304e-05 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685500+0000", 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005299242 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685514+0000", 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3461e-05 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.511 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:602 3.f e28/24 3:f4a2090e:::rbd_object_map.116011b0bca6:head v 28'21, mlcod=28'14)", 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.680604+0000", 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.8140066150000003, 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0049389439999999998, 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680604+0000", 2024-09-12T07:19:53.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680604+0000", 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680604+0000", 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6699999999999999e-07 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680608+0000", 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.428e-06 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680615+0000", 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6350000000000006e-06 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680620+0000", 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.267e-06 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680665+0000", 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5339000000000001e-05 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680684+0000", 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9449e-05 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685516+0000", 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0048312329999999999 2024-09-12T07:19:53.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685543+0000", 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7126000000000001e-05 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:612 3.e e28/24 3:74ddf403:::rbd_data.116011b0bca6.0000000000000008:head v 28'19, mlcod=28'17)", 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.686317+0000", 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.8082936390000004, 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0049658910000000001, 2024-09-12T07:19:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.686317+0000", 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.686317+0000", 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.686317+0000", 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3499999999999996e-07 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.686322+0000", 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2889999999999999e-06 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.686334+0000", 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.201e-05 2024-09-12T07:19:53.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.686340+0000", 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6059999999999999e-06 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.686387+0000", 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7034999999999999e-05 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.686409+0000", 2024-09-12T07:19:53.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1688e-05 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.691192+0000", 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0047835259999999998 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.691283+0000", 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0202000000000001e-05 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1331 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'20, mlcod=29'17)", 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.746281+0000", 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.7483292600000002, 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053784150000000001, 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746281+0000", 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.522 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.522 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746281+0000", 2024-09-12T07:19:53.522 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.522 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.522 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746282+0000", 2024-09-12T07:19:53.522 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7899999999999998e-07 2024-09-12T07:19:53.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.522 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.523 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746289+0000", 2024-09-12T07:19:53.523 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2690000000000002e-06 2024-09-12T07:19:53.523 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.523 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.523 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.523 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746301+0000", 2024-09-12T07:19:53.523 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1544e-05 2024-09-12T07:19:53.523 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.523 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.523 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.523 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746307+0000", 2024-09-12T07:19:53.523 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4060000000000002e-06 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746356+0000", 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9246999999999998e-05 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746371+0000", 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5101e-05 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.751626+0000", 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0052545680000000003 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.751660+0000", 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3600999999999999e-05 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.525 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1332 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'21, mlcod=29'17)", 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.747073+0000", 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.7475374749999997, 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0076436810000000003, 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747073+0000", 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.526 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747073+0000", 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747073+0000", 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4899999999999995e-07 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747077+0000", 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6270000000000002e-06 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.527 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747086+0000", 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.302e-06 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747091+0000", 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4839999999999998e-06 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747137+0000", 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5896000000000002e-05 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.528 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747157+0000", 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-05 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754641+0000", 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0074841810000000003 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754717+0000", 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5562000000000006e-05 2024-09-12T07:19:53.529 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1333 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'22, mlcod=29'17)", 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.747710+0000", 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.74690075, 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070469909999999998, 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.530 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747710+0000", 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747710+0000", 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747710+0000", 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.531 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747713+0000", 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1300000000000001e-06 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747720+0000", 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7179999999999999e-06 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747723+0000", 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5190000000000001e-06 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747773+0000", 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9549000000000003e-05 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747795+0000", 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2450999999999999e-05 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754722+0000", 2024-09-12T07:19:53.533 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0069261949999999996 2024-09-12T07:19:53.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.534 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754757+0000", 2024-09-12T07:19:53.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4995e-05 2024-09-12T07:19:53.534 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.534 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.534 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.534 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1334 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'23, mlcod=29'18)", 2024-09-12T07:19:53.534 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.748316+0000", 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.7462945400000001, 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0064707979999999998, 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748316+0000", 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748316+0000", 2024-09-12T07:19:53.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748316+0000", 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748320+0000", 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7699999999999999e-06 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.536 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748329+0000", 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4100000000000008e-06 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748333+0000", 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2729999999999996e-06 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748376+0000", 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3464000000000003e-05 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748398+0000", 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1348999999999998e-05 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754758+0000", 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0063606840000000001 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754787+0000", 2024-09-12T07:19:53.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8357999999999999e-05 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1335 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'24, mlcod=29'18)", 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.748830+0000", 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.7457799319999996, 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059871769999999998, 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.539 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748830+0000", 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748830+0000", 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748831+0000", 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7999999999999995e-07 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748835+0000", 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1139999999999999e-06 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748845+0000", 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4569999999999994e-06 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748849+0000", 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6009999999999997e-06 2024-09-12T07:19:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748901+0000", 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2280000000000001e-05 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748922+0000", 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0047999999999998e-05 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754788+0000", 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058663999999999999 2024-09-12T07:19:53.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754818+0000", 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9697e-05 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1336 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'25, mlcod=29'18)", 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.749482+0000", 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.7451285329999999, 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053627120000000004, 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.543 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749482+0000", 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749482+0000", 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749483+0000", 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6499999999999999e-07 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749488+0000", 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749500+0000", 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1914e-05 2024-09-12T07:19:53.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749508+0000", 2024-09-12T07:19:53.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9149999999999996e-06 2024-09-12T07:19:53.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749556+0000", 2024-09-12T07:19:53.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8766e-05 2024-09-12T07:19:53.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749578+0000", 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1500000000000001e-05 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754819+0000", 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0052410199999999999 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754845+0000", 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5752e-05 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.547 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1337 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'26, mlcod=29'18)", 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.750035+0000", 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.7445750039999997, 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0048421569999999997, 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.750035+0000", 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.750035+0000", 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.750036+0000", 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2799999999999996e-07 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.750040+0000", 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1500000000000001e-06 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.750049+0000", 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9730000000000003e-06 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.750053+0000", 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3320000000000002e-06 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.750103+0000", 2024-09-12T07:19:53.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9308000000000003e-05 2024-09-12T07:19:53.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.750126+0000", 2024-09-12T07:19:53.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2986999999999999e-05 2024-09-12T07:19:53.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754846+0000", 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0047199729999999997 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754878+0000", 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1906000000000003e-05 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.552 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1345 3.9 e29/24 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:head v 29'49, mlcod=29'47)", 2024-09-12T07:19:53.553 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.756989+0000", 2024-09-12T07:19:53.553 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.7376212970000005, 2024-09-12T07:19:53.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0052040960000000001, 2024-09-12T07:19:53.553 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.553 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.553 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.553 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.756989+0000", 2024-09-12T07:19:53.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.756989+0000", 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.756990+0000", 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8500000000000001e-07 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.758150+0000", 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001160283 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.758161+0000", 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0723999999999999e-05 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.758166+0000", 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4489999999999998e-06 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.758209+0000", 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2772000000000003e-05 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.758223+0000", 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4477e-05 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.762163+0000", 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039393939999999997 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.762193+0000", 2024-09-12T07:19:53.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0412e-05 2024-09-12T07:19:53.557 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.557 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.557 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.557 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1665 3.1 e30/24 3:859610a1:::rbd_header.1160947fa40f:head v 30'5, mlcod=30'3)", 2024-09-12T07:19:53.557 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:49.532407+0000", 2024-09-12T07:19:53.557 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 3.962203701, 2024-09-12T07:19:53.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0052828839999999998, 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.532407+0000", 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.532407+0000", 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.532407+0000", 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3e-07 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.532412+0000", 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0010000000000001e-06 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.532422+0000", 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7389999999999994e-06 2024-09-12T07:19:53.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.532427+0000", 2024-09-12T07:19:53.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6e-06 2024-09-12T07:19:53.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.532468+0000", 2024-09-12T07:19:53.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1560000000000002e-05 2024-09-12T07:19:53.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.532480+0000", 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1685000000000001e-05 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.537560+0000", 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050804609999999997 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.537690+0000", 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012920799999999999 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.561 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1687 3.1c e30/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 30'71, mlcod=30'68)", 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:49.557056+0000", 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 3.9375547979999999, 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006603757, 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557056+0000", 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557056+0000", 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557056+0000", 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0200000000000002e-07 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557062+0000", 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9850000000000003e-06 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557071+0000", 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1770000000000004e-06 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557076+0000", 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4689999999999999e-06 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557117+0000", 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1189999999999997e-05 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557134+0000", 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6637999999999999e-05 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563625+0000", 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0064912679999999997 2024-09-12T07:19:53.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563659+0000", 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4428000000000003e-05 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1688 3.1c e30/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 30'72, mlcod=30'68)", 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:49.557723+0000", 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 3.9368873259999999, 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059720429999999998, 2024-09-12T07:19:53.566 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557723+0000", 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557723+0000", 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557724+0000", 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7599999999999996e-07 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557730+0000", 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1290000000000001e-06 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557744+0000", 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3862999999999999e-05 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557751+0000", 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4070000000000004e-06 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557806+0000", 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4379000000000002e-05 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557827+0000", 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1410999999999999e-05 2024-09-12T07:19:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563662+0000", 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058354230000000002 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563695+0000", 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2654999999999998e-05 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1689 3.1c e30/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 30'73, mlcod=30'68)", 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:49.558399+0000", 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 3.936211648, 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053262459999999998, 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558399+0000", 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558399+0000", 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558399+0000", 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.75e-07 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558405+0000", 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.259e-06 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558417+0000", 2024-09-12T07:19:53.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1963e-05 2024-09-12T07:19:53.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558423+0000", 2024-09-12T07:19:53.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5139999999999998e-06 2024-09-12T07:19:53.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558484+0000", 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0717999999999997e-05 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558507+0000", 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2858e-05 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563697+0000", 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0051899340000000002 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563725+0000", 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8325000000000001e-05 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1690 3.1c e30/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 30'74, mlcod=30'68)", 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:49.558964+0000", 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 3.9356464579999999, 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0048281729999999998, 2024-09-12T07:19:53.575 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558964+0000", 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558964+0000", 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558965+0000", 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6599999999999995e-07 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558970+0000", 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6710000000000004e-06 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558982+0000", 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1722999999999999e-05 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.558989+0000", 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2389999999999996e-06 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.559041+0000", 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1867000000000003e-05 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.559065+0000", 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3512000000000001e-05 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563726+0000", 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0046614969999999997 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563792+0000", 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5897999999999998e-05 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:2067 3.1d e30/24 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:head v 30'14, mlcod=28'12)", 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:49.858445+0000", 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 3.636164924, 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059962629999999999, 2024-09-12T07:19:53.579 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.858445+0000", 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.858445+0000", 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.858446+0000", 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3600000000000003e-07 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859175+0000", 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00072900199999999997 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859191+0000", 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5376e-05 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859198+0000", 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4989999999999997e-06 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859263+0000", 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4519999999999999e-05 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859281+0000", 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7971000000000001e-05 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.864380+0000", 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050996720000000004 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.864442+0000", 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1487000000000003e-05 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:53.583 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:53.585 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone4 from 0xba24625 (overlap 0x1b4d7bb) to 0x131ca36 2024-09-12T07:19:53.586 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:19:53.586 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:53.586+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:53.685 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:53.687+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:53.748 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:19:53.748 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:53.757 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:19:53.800 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:53.802+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:53.888 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:53.889+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:53.909 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:19:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:19:53.909 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:53.921 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:19:54.007 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:54.009+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:54.089 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:54.091+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:54.130 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:19:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:19:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:19:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:7769 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7452672~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.956783+0000", 2024-09-12T07:19:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.17332794000000001, 2024-09-12T07:19:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052361474999999998, 2024-09-12T07:19:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.130 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7769 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956783+0000", 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956783+0000", 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956784+0000", 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5900000000000002e-07 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956786+0000", 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.018e-06 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956803+0000", 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7847e-05 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956809+0000", 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8839999999999999e-06 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.008926+0000", 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05211673 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.009021+0000", 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5099999999999994e-05 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.009145+0000", 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000123437 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.134 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:7771 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7457792~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.956822+0000", 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.17328853399999999, 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052560981, 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7771 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956822+0000", 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956822+0000", 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.961941+0000", 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0051181120000000002 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.961946+0000", 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9110000000000001e-06 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.961960+0000", 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4545999999999999e-05 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.961968+0000", 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:19:54.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.009180+0000", 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.047212744000000001 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.009274+0000", 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4186999999999998e-05 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.009383+0000", 2024-09-12T07:19:54.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109027 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:7819 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7565312~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.962546+0000", 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.167564772, 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052269815999999997, 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7819 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962546+0000", 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962546+0000", 2024-09-12T07:19:54.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962547+0000", 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962549+0000", 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2340000000000001e-06 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962557+0000", 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3350000000000007e-06 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962564+0000", 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7669999999999998e-06 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.014624+0000", 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052060141999999997 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.014721+0000", 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6410999999999996e-05 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.014816+0000", 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5494000000000004e-05 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.143 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:7821 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7570432~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.962573+0000", 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.16753811199999999, 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052477237000000003, 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7821 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962573+0000", 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962573+0000", 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962573+0000", 2024-09-12T07:19:54.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3000000000000001e-07 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962575+0000", 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1600000000000001e-06 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962584+0000", 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.225e-06 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962589+0000", 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.383e-06 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.014850+0000", 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052260747000000003 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.014939+0000", 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9600999999999998e-05 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015050+0000", 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000110691 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:7823 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7575552~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.962598+0000", 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.16751302100000001, 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052623057000000001, 2024-09-12T07:19:54.148 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7823 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962598+0000", 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962598+0000", 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962598+0000", 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5699999999999998e-07 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962601+0000", 2024-09-12T07:19:54.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0820000000000001e-06 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962609+0000", 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0749999999999998e-06 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962613+0000", 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2740000000000001e-06 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015086+0000", 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052473335000000003 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015170+0000", 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3386000000000004e-05 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015221+0000", 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1447999999999999e-05 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.152 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:7826 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7580672~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.962618+0000", 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.167492905, 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052727082000000002, 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.153 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7826 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962618+0000", 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962618+0000", 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.967976+0000", 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053580490000000001 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.967981+0000", 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0780000000000003e-06 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.967991+0000", 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4450000000000002e-06 2024-09-12T07:19:54.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.967997+0000", 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6969999999999996e-06 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015255+0000", 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.047257368000000001 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015297+0000", 2024-09-12T07:19:54.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2073999999999997e-05 2024-09-12T07:19:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015345+0000", 2024-09-12T07:19:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8371000000000002e-05 2024-09-12T07:19:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.157 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8222 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 8151040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.026213+0000", 2024-09-12T07:19:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.103898358, 2024-09-12T07:19:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052383225999999998, 2024-09-12T07:19:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8222 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026213+0000", 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026213+0000", 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026213+0000", 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7700000000000004e-07 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026216+0000", 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5830000000000001e-06 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026223+0000", 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4059999999999999e-06 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026229+0000", 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5690000000000004e-06 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078402+0000", 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052172812999999998 2024-09-12T07:19:54.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078496+0000", 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4393000000000002e-05 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078596+0000", 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9884999999999997e-05 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8223 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 6339584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.026236+0000", 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.10387511000000001, 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052592912999999998, 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8223 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.167 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026236+0000", 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026236+0000", 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026317+0000", 2024-09-12T07:19:54.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0649999999999995e-05 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026320+0000", 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0709999999999999e-06 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026328+0000", 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2660000000000001e-06 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026334+0000", 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1829999999999999e-06 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078630+0000", 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052296294 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078722+0000", 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1138999999999996e-05 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078829+0000", 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010731 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8224 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 8156160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.026345+0000", 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.103766383, 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052664641999999998, 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.171 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8224 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026345+0000", 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026345+0000", 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.031887+0000", 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055426449999999997 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.031891+0000", 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2080000000000002e-06 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.031902+0000", 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.066e-05 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.031910+0000", 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5299999999999999e-06 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078885+0000", 2024-09-12T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.046975544000000001 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078943+0000", 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7334999999999999e-05 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.079009+0000", 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6719999999999998e-05 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8392 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 6774784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.052760+0000", 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.077351342000000003, 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052240091000000002, 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8392 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.052760+0000", 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.052760+0000", 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.058011+0000", 2024-09-12T07:19:54.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00525181 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.058017+0000", 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0719999999999999e-06 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.058030+0000", 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3915e-05 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.058037+0000", 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1090000000000003e-06 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.104765+0000", 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.046728891000000002 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.104879+0000", 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113256 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.105000+0000", 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000121038 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8445 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 6948864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.059945+0000", 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.070165951000000004, 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052372780000000001, 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.180 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8445 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059945+0000", 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059945+0000", 2024-09-12T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059946+0000", 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2499999999999995e-07 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059948+0000", 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7599999999999998e-06 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059956+0000", 2024-09-12T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8269999999999998e-06 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059962+0000", 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7420000000000003e-06 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.112121+0000", 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052159097000000001 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.112213+0000", 2024-09-12T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1531000000000002e-05 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.112318+0000", 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010529799999999999 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8446 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.059970+0000", 2024-09-12T07:19:54.184 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.070141242000000006, 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052538337999999997, 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8446 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059970+0000", 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059970+0000", 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066075+0000", 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061051339999999999 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066079+0000", 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7950000000000001e-06 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066089+0000", 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0078000000000001e-05 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066094+0000", 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0679999999999996e-06 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.112345+0000", 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.046251249000000001 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.112420+0000", 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5246000000000004e-05 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.112508+0000", 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7768000000000004e-05 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.188 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8477 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7030784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.066789+0000", 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.063322078000000004, 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052337458000000003, 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8477 2024-09-12T07:19:54.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066789+0000", 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066789+0000", 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066789+0000", 2024-09-12T07:19:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066792+0000", 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4219999999999999e-06 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066799+0000", 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7100000000000001e-06 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066803+0000", 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9080000000000003e-06 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.118930+0000", 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052126611000000003 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119023+0000", 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2851999999999998e-05 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119126+0000", 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010345599999999999 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8478 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7035904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.066809+0000", 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.063302368999999997, 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052560395000000003, 2024-09-12T07:19:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8478 2024-09-12T07:19:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066809+0000", 2024-09-12T07:19:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066809+0000", 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066809+0000", 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8999999999999998e-07 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066810+0000", 2024-09-12T07:19:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5179999999999999e-06 2024-09-12T07:19:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066814+0000", 2024-09-12T07:19:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8929999999999998e-06 2024-09-12T07:19:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066817+0000", 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6579999999999998e-06 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119172+0000", 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05235542 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119261+0000", 2024-09-12T07:19:54.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8315999999999999e-05 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119369+0000", 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0001083 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8479 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7426048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.066820+0000", 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.063290571000000004, 2024-09-12T07:19:54.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052787775000000002, 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8479 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066820+0000", 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066820+0000", 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.071929+0000", 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005108334 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.071934+0000", 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0220000000000004e-06 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.071953+0000", 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8811e-05 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.071962+0000", 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3570000000000004e-06 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119414+0000", 2024-09-12T07:19:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.047451568999999999 2024-09-12T07:19:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119505+0000", 2024-09-12T07:19:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1061000000000001e-05 2024-09-12T07:19:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119608+0000", 2024-09-12T07:19:54.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000103621 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8502 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7482368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.072395+0000", 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.057715611, 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052348634999999998, 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8502 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072395+0000", 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072395+0000", 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072396+0000", 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9500000000000002e-07 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072399+0000", 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072407+0000", 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.96e-06 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072413+0000", 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.246e-06 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.124598+0000", 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052185441999999999 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.124665+0000", 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6960999999999998e-05 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.124744+0000", 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8786000000000001e-05 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.207 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8503 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7102464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.072422+0000", 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.057689437000000003, 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052572356000000001, 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8503 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072422+0000", 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072422+0000", 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072422+0000", 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3300000000000002e-07 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072424+0000", 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3209999999999999e-06 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072433+0000", 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3410000000000003e-06 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072438+0000", 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7039999999999999e-06 2024-09-12T07:19:54.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.124783+0000", 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052344684000000002 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.124880+0000", 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7226999999999996e-05 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.124994+0000", 2024-09-12T07:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113546 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8504 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7487488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.072446+0000", 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.057665322999999997, 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052688156, 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.212 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8504 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072446+0000", 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072446+0000", 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072446+0000", 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6499999999999999e-07 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072448+0000", 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1320000000000001e-06 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072457+0000", 2024-09-12T07:19:54.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9590000000000001e-06 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072463+0000", 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4739999999999996e-06 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125031+0000", 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052567908000000003 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125076+0000", 2024-09-12T07:19:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5569999999999999e-05 2024-09-12T07:19:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125134+0000", 2024-09-12T07:19:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7648000000000001e-05 2024-09-12T07:19:54.216 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8505 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7107584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.072470+0000", 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.057640935999999997, 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052904026, 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8505 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072470+0000", 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072470+0000", 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072471+0000", 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5400000000000002e-07 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072473+0000", 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1880000000000001e-06 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072480+0000", 2024-09-12T07:19:54.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2459999999999997e-06 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072487+0000", 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1189999999999999e-06 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125176+0000", 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052689034000000003 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125271+0000", 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4590000000000001e-05 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125374+0000", 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000103395 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7492608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.221 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.072494+0000", 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.057616645000000001, 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.053008686999999999, 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8506 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072494+0000", 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072494+0000", 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078003+0000", 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055090909999999998 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078008+0000", 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2899999999999996e-06 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078018+0000", 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.006e-05 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078025+0000", 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7560000000000002e-06 2024-09-12T07:19:54.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125404+0000", 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.047379457999999999 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125448+0000", 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3628999999999999e-05 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125503+0000", 2024-09-12T07:19:54.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5402999999999998e-05 2024-09-12T07:19:54.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.226 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.226 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.226 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:54.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:19:54.228 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:19:54.229 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:54.229+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:54.328 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:54.330+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:54.355 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:19:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:19:54.373 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:19:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:19:54.391 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:19:54.391 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:19:54.408 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11300 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.369731+0000", 2024-09-12T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025361938000000001, 2024-09-12T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025383912000000002, 2024-09-12T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11300 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369731+0000", 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369731+0000", 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369731+0000", 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369733+0000", 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9810000000000002e-06 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369739+0000", 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1619999999999996e-06 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369744+0000", 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8550000000000001e-06 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11332 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376121+0000", 2024-09-12T07:19:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018971238000000001, 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019066013999999999, 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11332 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376121+0000", 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376121+0000", 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376122+0000", 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8099999999999998e-07 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376124+0000", 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.311e-06 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376130+0000", 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4339999999999997e-06 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376135+0000", 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0080000000000002e-06 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11364 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376829+0000", 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018263613000000001, 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018411090000000001, 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11364 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376829+0000", 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376829+0000", 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376829+0000", 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3700000000000001e-07 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376831+0000", 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.154e-06 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376837+0000", 2024-09-12T07:19:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9270000000000002e-06 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376842+0000", 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6820000000000002e-06 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11396 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383243+0000", 2024-09-12T07:19:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011849821, 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01204762, 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11396 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383243+0000", 2024-09-12T07:19:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383243+0000", 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383243+0000", 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2999999999999999e-07 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383244+0000", 2024-09-12T07:19:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0979999999999999e-06 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383248+0000", 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6610000000000002e-06 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383250+0000", 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7580000000000001e-06 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11428 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383931+0000", 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011161179, 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011408223, 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11428 2024-09-12T07:19:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383931+0000", 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383931+0000", 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383932+0000", 2024-09-12T07:19:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1e-07 2024-09-12T07:19:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383933+0000", 2024-09-12T07:19:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1000000000000001e-06 2024-09-12T07:19:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383936+0000", 2024-09-12T07:19:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.168e-06 2024-09-12T07:19:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383938+0000", 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5840000000000002e-06 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11460 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390585+0000", 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.004507222, 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0048057880000000001, 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11460 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390585+0000", 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390585+0000", 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390586+0000", 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4900000000000001e-07 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390588+0000", 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1619999999999998e-06 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390594+0000", 2024-09-12T07:19:54.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.567e-06 2024-09-12T07:19:54.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390599+0000", 2024-09-12T07:19:54.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7999999999999998e-06 2024-09-12T07:19:54.429 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.429 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.429 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11492 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391318+0000", 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0037745859999999999, 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0041218499999999998, 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11492 2024-09-12T07:19:54.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391318+0000", 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391318+0000", 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391318+0000", 2024-09-12T07:19:54.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8099999999999998e-07 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391320+0000", 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.978e-06 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391326+0000", 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8799999999999996e-06 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391331+0000", 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.848e-06 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11301 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.369751+0000", 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025341546999999999, 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025738530999999999, 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.433 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11301 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369751+0000", 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369751+0000", 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369751+0000", 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7e-07 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369754+0000", 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2850000000000001e-06 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369760+0000", 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1959999999999996e-06 2024-09-12T07:19:54.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369765+0000", 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9130000000000002e-06 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11333 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376143+0000", 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.01894941, 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019398735, 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.436 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.437 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.437 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.437 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.437 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.437 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11333 2024-09-12T07:19:54.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.437 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376143+0000", 2024-09-12T07:19:54.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376143+0000", 2024-09-12T07:19:54.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376143+0000", 2024-09-12T07:19:54.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8099999999999998e-07 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376146+0000", 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1500000000000002e-06 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376152+0000", 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6279999999999999e-06 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376157+0000", 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2120000000000002e-06 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11365 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376848+0000", 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018244316999999999, 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018744567, 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.440 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11365 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376848+0000", 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376848+0000", 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376848+0000", 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6199999999999999e-07 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376851+0000", 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1320000000000001e-06 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376856+0000", 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8710000000000002e-06 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376861+0000", 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8969999999999999e-06 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11397 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.443 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383254+0000", 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011838262, 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012390071000000001, 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11397 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383254+0000", 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383254+0000", 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383254+0000", 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2700000000000001e-07 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383255+0000", 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9899999999999999e-07 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383258+0000", 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.993e-06 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383261+0000", 2024-09-12T07:19:54.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6639999999999998e-06 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11429 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383942+0000", 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011150676, 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011749994, 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.447 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11429 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383942+0000", 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383942+0000", 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383942+0000", 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.28e-07 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383950+0000", 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8990000000000001e-06 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383954+0000", 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8130000000000002e-06 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383957+0000", 2024-09-12T07:19:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.886e-06 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11461 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390606+0000", 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0044867880000000002, 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0051356170000000003, 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11461 2024-09-12T07:19:54.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.451 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390606+0000", 2024-09-12T07:19:54.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390606+0000", 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390606+0000", 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3099999999999998e-07 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390608+0000", 2024-09-12T07:19:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390614+0000", 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0390000000000001e-06 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390619+0000", 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9300000000000002e-06 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.453 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11493 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391337+0000", 2024-09-12T07:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0037553460000000001, 2024-09-12T07:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0044525830000000004, 2024-09-12T07:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11493 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391337+0000", 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391337+0000", 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391337+0000", 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391340+0000", 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3609999999999999e-06 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391346+0000", 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:19:54.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391351+0000", 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9180000000000002e-06 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11302 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.369773+0000", 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025319327999999999, 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026066405000000001, 2024-09-12T07:19:54.457 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11302 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369773+0000", 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369773+0000", 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369774+0000", 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3200000000000005e-07 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369776+0000", 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0470000000000001e-06 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369782+0000", 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8359999999999997e-06 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369787+0000", 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1200000000000001e-06 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11334 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376164+0000", 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018928647, 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019739929, 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11334 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376164+0000", 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376164+0000", 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376164+0000", 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:19:54.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376171+0000", 2024-09-12T07:19:54.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9840000000000004e-06 2024-09-12T07:19:54.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376178+0000", 2024-09-12T07:19:54.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3319999999999997e-06 2024-09-12T07:19:54.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376183+0000", 2024-09-12T07:19:54.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6169999999999998e-06 2024-09-12T07:19:54.464 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.464 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.464 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.464 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11366 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.464 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376877+0000", 2024-09-12T07:19:54.464 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018215568000000001, 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019077561, 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11366 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376877+0000", 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376877+0000", 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376877+0000", 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4700000000000002e-07 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376880+0000", 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4229999999999999e-06 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376886+0000", 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7329999999999997e-06 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376892+0000", 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4210000000000003e-06 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.467 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11398 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383264+0000", 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011828297999999999, 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012766779000000001, 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11398 2024-09-12T07:19:54.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383264+0000", 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383264+0000", 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383264+0000", 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2000000000000001e-07 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383266+0000", 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1489999999999999e-06 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383268+0000", 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9799999999999998e-06 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383271+0000", 2024-09-12T07:19:54.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5940000000000001e-06 2024-09-12T07:19:54.471 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.471 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.471 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.471 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11430 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.471 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383961+0000", 2024-09-12T07:19:54.471 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011131927, 2024-09-12T07:19:54.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012122236999999999, 2024-09-12T07:19:54.471 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11430 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383961+0000", 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.472 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383961+0000", 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383961+0000", 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8000000000000002e-07 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383962+0000", 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.116e-06 2024-09-12T07:19:54.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383965+0000", 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9529999999999999e-06 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383967+0000", 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5730000000000002e-06 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.474 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11462 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.475 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390625+0000", 2024-09-12T07:19:54.475 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0044671160000000001, 2024-09-12T07:19:54.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055105629999999996, 2024-09-12T07:19:54.475 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.475 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.475 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.475 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.475 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.475 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.475 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11462 2024-09-12T07:19:54.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.475 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390625+0000", 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390625+0000", 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390626+0000", 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7899999999999999e-07 2024-09-12T07:19:54.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390628+0000", 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0059999999999999e-06 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390634+0000", 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3659999999999997e-06 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390639+0000", 2024-09-12T07:19:54.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0509999999999996e-06 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11494 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391368+0000", 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0037244000000000001, 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0048167319999999998, 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.478 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11494 2024-09-12T07:19:54.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.479 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391368+0000", 2024-09-12T07:19:54.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391368+0000", 2024-09-12T07:19:54.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391369+0000", 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1799999999999995e-07 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391371+0000", 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4049999999999998e-06 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391378+0000", 2024-09-12T07:19:54.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0310000000000001e-06 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391384+0000", 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4949999999999999e-06 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11303 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.369795+0000", 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025297878999999999, 2024-09-12T07:19:54.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026441175000000001, 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11303 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369795+0000", 2024-09-12T07:19:54.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369795+0000", 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369795+0000", 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369797+0000", 2024-09-12T07:19:54.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1770000000000001e-06 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369803+0000", 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9089999999999997e-06 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369808+0000", 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7589999999999997e-06 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.485 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11335 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.485 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376190+0000", 2024-09-12T07:19:54.485 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018902229, 2024-09-12T07:19:54.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020098995000000001, 2024-09-12T07:19:54.485 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.485 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.485 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.485 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.485 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.485 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.485 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11335 2024-09-12T07:19:54.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.486 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376190+0000", 2024-09-12T07:19:54.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376190+0000", 2024-09-12T07:19:54.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376191+0000", 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8800000000000002e-07 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376193+0000", 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2369999999999999e-06 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376199+0000", 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1820000000000003e-06 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376204+0000", 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9579999999999998e-06 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11367 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376899+0000", 2024-09-12T07:19:54.488 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018193417999999999, 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019439444, 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11367 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376899+0000", 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376899+0000", 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376899+0000", 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.41e-07 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376902+0000", 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1610000000000001e-06 2024-09-12T07:19:54.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376908+0000", 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4150000000000004e-06 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376913+0000", 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.763e-06 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.491 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11399 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383274+0000", 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011818093999999999, 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0131182, 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11399 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383274+0000", 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383274+0000", 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383275+0000", 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2999999999999999e-07 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383276+0000", 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.057e-06 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383279+0000", 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0290000000000001e-06 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383281+0000", 2024-09-12T07:19:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6089999999999999e-06 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11431 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383970+0000", 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011122, 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012471955999999999, 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11431 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383970+0000", 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383970+0000", 2024-09-12T07:19:54.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383971+0000", 2024-09-12T07:19:54.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2999999999999999e-07 2024-09-12T07:19:54.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383972+0000", 2024-09-12T07:19:54.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0130000000000001e-06 2024-09-12T07:19:54.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383975+0000", 2024-09-12T07:19:54.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8789999999999999e-06 2024-09-12T07:19:54.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383978+0000", 2024-09-12T07:19:54.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3859999999999999e-06 2024-09-12T07:19:54.498 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11463 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390645+0000", 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0044469640000000003, 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058495350000000003, 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.499 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11463 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390645+0000", 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390645+0000", 2024-09-12T07:19:54.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390646+0000", 2024-09-12T07:19:54.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:19:54.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390648+0000", 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2709999999999999e-06 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390654+0000", 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2729999999999999e-06 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390659+0000", 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8010000000000003e-06 2024-09-12T07:19:54.502 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11495 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391391+0000", 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0037017320000000001, 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0051533660000000004, 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.503 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11495 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391391+0000", 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391391+0000", 2024-09-12T07:19:54.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391505+0000", 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011459399999999999 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391509+0000", 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6069999999999999e-06 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391518+0000", 2024-09-12T07:19:54.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8519999999999993e-06 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391524+0000", 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4060000000000002e-06 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11304 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.369815+0000", 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025277877000000001, 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026782909000000001, 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11304 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369815+0000", 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369815+0000", 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369815+0000", 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8799999999999998e-07 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369817+0000", 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0779999999999998e-06 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369823+0000", 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8619999999999999e-06 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369828+0000", 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9080000000000003e-06 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11336 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.509 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376211+0000", 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018881638999999999, 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020436843999999999, 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11336 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376211+0000", 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376211+0000", 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376211+0000", 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376213+0000", 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.99e-06 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376220+0000", 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3770000000000001e-06 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376225+0000", 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9690000000000002e-06 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.512 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11368 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376921+0000", 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018171129000000001, 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019779501000000001, 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11368 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.513 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376921+0000", 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376921+0000", 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376922+0000", 2024-09-12T07:19:54.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9500000000000003e-07 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376924+0000", 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.249e-06 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376930+0000", 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8109999999999999e-06 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376935+0000", 2024-09-12T07:19:54.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1950000000000002e-06 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11400 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383284+0000", 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011808002999999999, 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013466084999999999, 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.516 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11400 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383284+0000", 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383284+0000", 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383285+0000", 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4999999999999999e-07 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383286+0000", 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0160000000000001e-06 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383289+0000", 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2389999999999998e-06 2024-09-12T07:19:54.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383294+0000", 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5560000000000001e-06 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11432 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383981+0000", 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011111236, 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012822962, 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.519 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11432 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383981+0000", 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383981+0000", 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383981+0000", 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1899999999999999e-07 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383982+0000", 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3799999999999996e-07 2024-09-12T07:19:54.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383985+0000", 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.864e-06 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383988+0000", 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5679999999999998e-06 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11464 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390665+0000", 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0044269670000000004, 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006188252, 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11464 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390665+0000", 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390665+0000", 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390666+0000", 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390668+0000", 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8959999999999999e-06 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390674+0000", 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.314e-06 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390681+0000", 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7769999999999997e-06 2024-09-12T07:19:54.525 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11305 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.369834+0000", 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025258521999999999, 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027117783999999999, 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11305 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369834+0000", 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369834+0000", 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369834+0000", 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5199999999999998e-07 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369836+0000", 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9410000000000001e-06 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369843+0000", 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2999999999999998e-06 2024-09-12T07:19:54.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369847+0000", 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7400000000000004e-06 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11337 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376231+0000", 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018861793000000002, 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020778646000000001, 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11337 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376231+0000", 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376231+0000", 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376231+0000", 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2300000000000002e-07 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376233+0000", 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.125e-06 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376240+0000", 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4150000000000004e-06 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376244+0000", 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8770000000000001e-06 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11369 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376942+0000", 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018150755000000001, 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020122061, 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11369 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376942+0000", 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376942+0000", 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376942+0000", 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.63e-07 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376944+0000", 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1390000000000002e-06 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376950+0000", 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0109999999999997e-06 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376955+0000", 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11401 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383314+0000", 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011778619000000001, 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013800150000000001, 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11401 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.536 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383314+0000", 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383314+0000", 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383314+0000", 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6499999999999999e-07 2024-09-12T07:19:54.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383317+0000", 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3580000000000001e-06 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383324+0000", 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7789999999999998e-06 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383333+0000", 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6970000000000005e-06 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.538 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11433 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383991+0000", 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011101575000000001, 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013172437, 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11433 2024-09-12T07:19:54.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383991+0000", 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383991+0000", 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383991+0000", 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.23e-07 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383992+0000", 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.02e-06 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383995+0000", 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9229999999999998e-06 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383998+0000", 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.582e-06 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11465 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390688+0000", 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0044048560000000004, 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0065306189999999997, 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.542 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11465 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390688+0000", 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390688+0000", 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390688+0000", 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.77e-07 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390695+0000", 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5599999999999999e-06 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390701+0000", 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3949999999999998e-06 2024-09-12T07:19:54.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390706+0000", 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0390000000000004e-06 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11306 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.369854+0000", 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025238723000000001, 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027416950999999998, 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.545 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11306 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369854+0000", 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369854+0000", 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369854+0000", 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.72e-07 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369856+0000", 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0320000000000002e-06 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369862+0000", 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6860000000000003e-06 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369872+0000", 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0002e-05 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11338 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376251+0000", 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018841711000000001, 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.021073096999999999, 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.548 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11338 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376251+0000", 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376251+0000", 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376251+0000", 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6899999999999998e-07 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376253+0000", 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9099999999999999e-06 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376259+0000", 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1079999999999998e-06 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376264+0000", 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0209999999999999e-06 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11370 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376969+0000", 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018123535999999999, 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020406703000000002, 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.551 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11370 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376969+0000", 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376969+0000", 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376969+0000", 2024-09-12T07:19:54.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4300000000000003e-07 2024-09-12T07:19:54.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376972+0000", 2024-09-12T07:19:54.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6560000000000001e-06 2024-09-12T07:19:54.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376979+0000", 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8310000000000003e-06 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376984+0000", 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0579999999999997e-06 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11402 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383343+0000", 2024-09-12T07:19:54.554 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011748987000000001, 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014084681, 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11402 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383343+0000", 2024-09-12T07:19:54.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383343+0000", 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383344+0000", 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383346+0000", 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2879999999999999e-06 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383353+0000", 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6950000000000003e-06 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383363+0000", 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7079999999999992e-06 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.557 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11434 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.384001+0000", 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011091772, 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013477437, 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.558 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11434 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.384001+0000", 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.384001+0000", 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.384056+0000", 2024-09-12T07:19:54.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4979000000000003e-05 2024-09-12T07:19:54.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.384059+0000", 2024-09-12T07:19:54.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1750000000000001e-06 2024-09-12T07:19:54.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.384069+0000", 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0351e-05 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.384075+0000", 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1530000000000002e-06 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11466 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390725+0000", 2024-09-12T07:19:54.561 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0043677150000000003, 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0068067589999999999, 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11466 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390725+0000", 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390725+0000", 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390725+0000", 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1600000000000001e-07 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390728+0000", 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3039999999999999e-06 2024-09-12T07:19:54.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390735+0000", 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1679999999999998e-06 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390741+0000", 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0070000000000003e-06 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11307 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.369895+0000", 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025197588999999999, 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027688263000000001, 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11307 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369895+0000", 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369895+0000", 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369895+0000", 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1799999999999995e-07 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369898+0000", 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5679999999999998e-06 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369905+0000", 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4410000000000004e-06 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369911+0000", 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8510000000000004e-06 2024-09-12T07:19:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11339 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376271+0000", 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018821864000000001, 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.021367232999999999, 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.568 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11339 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376271+0000", 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376271+0000", 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376271+0000", 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5400000000000002e-07 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376273+0000", 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.34e-06 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376279+0000", 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0249999999999999e-06 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376284+0000", 2024-09-12T07:19:54.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1490000000000002e-06 2024-09-12T07:19:54.571 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.571 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.571 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.571 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11371 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.571 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376992+0000", 2024-09-12T07:19:54.571 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018100774, 2024-09-12T07:19:54.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020696776, 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11371 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376992+0000", 2024-09-12T07:19:54.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376992+0000", 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376992+0000", 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376994+0000", 2024-09-12T07:19:54.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0789999999999999e-06 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377001+0000", 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8850000000000001e-06 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377006+0000", 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1510000000000003e-06 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11403 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383381+0000", 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011711908, 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014361311999999999, 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11403 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383381+0000", 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383381+0000", 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383381+0000", 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7800000000000002e-07 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383383+0000", 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.384e-06 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383391+0000", 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1359999999999999e-06 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383396+0000", 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3430000000000003e-06 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11435 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.384086+0000", 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011006692, 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013705692, 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11435 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.384086+0000", 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.384086+0000", 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.389959+0000", 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058732289999999998 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.389964+0000", 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7480000000000001e-06 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.389976+0000", 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2479e-05 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.389984+0000", 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9529999999999999e-06 2024-09-12T07:19:54.580 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11467 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390749+0000", 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.004343469, 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.007095708, 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.581 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11467 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390749+0000", 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390749+0000", 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390749+0000", 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1500000000000005e-07 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390752+0000", 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1299999999999999e-06 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390758+0000", 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5080000000000002e-06 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390763+0000", 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0379999999999999e-06 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11308 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.369919+0000", 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025173253, 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027981834000000001, 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.584 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11308 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369919+0000", 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369919+0000", 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369920+0000", 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9999999999999998e-07 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369922+0000", 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3309999999999998e-06 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369928+0000", 2024-09-12T07:19:54.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9699999999999996e-06 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369933+0000", 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.933e-06 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11340 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376291+0000", 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018801766000000001, 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.021664950999999998, 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11340 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376291+0000", 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376291+0000", 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376291+0000", 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5499999999999998e-07 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376293+0000", 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.255e-06 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376300+0000", 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1829999999999999e-06 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376306+0000", 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9279999999999998e-06 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11372 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377013+0000", 2024-09-12T07:19:54.590 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018079799000000001, 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020993312, 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11372 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377013+0000", 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377013+0000", 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377013+0000", 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377015+0000", 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0940000000000002e-06 2024-09-12T07:19:54.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377021+0000", 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9399999999999999e-06 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377026+0000", 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7219999999999999e-06 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11404 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383403+0000", 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011689098, 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014654083, 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11404 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383403+0000", 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383403+0000", 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383404+0000", 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6199999999999998e-07 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383406+0000", 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.091e-06 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383415+0000", 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6449999999999994e-06 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383421+0000", 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2790000000000004e-06 2024-09-12T07:19:54.596 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.597 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.597 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.597 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11436 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.597 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.389998+0000", 2024-09-12T07:19:54.597 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0050939949999999996, 2024-09-12T07:19:54.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0081121189999999992, 2024-09-12T07:19:54.597 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.597 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.597 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.597 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.598 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.598 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.598 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11436 2024-09-12T07:19:54.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.598 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.389998+0000", 2024-09-12T07:19:54.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.389998+0000", 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.389999+0000", 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9500000000000002e-07 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390002+0000", 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8140000000000001e-06 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390009+0000", 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.306e-06 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390015+0000", 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0599999999999996e-06 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.600 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11468 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390769+0000", 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0043231789999999999, 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0073907390000000003, 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.601 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11468 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390769+0000", 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390769+0000", 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390770+0000", 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5199999999999998e-07 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390772+0000", 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2819999999999999e-06 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390778+0000", 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2450000000000003e-06 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390783+0000", 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7550000000000002e-06 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11309 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.369940+0000", 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025152833999999999, 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028274647999999999, 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.605 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.605 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.605 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.605 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.605 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11309 2024-09-12T07:19:54.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.605 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369940+0000", 2024-09-12T07:19:54.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369940+0000", 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369940+0000", 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369942+0000", 2024-09-12T07:19:54.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2390000000000001e-06 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369948+0000", 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369953+0000", 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7999999999999998e-06 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11341 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376312+0000", 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.01878032, 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0219548, 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11341 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376312+0000", 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376312+0000", 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376313+0000", 2024-09-12T07:19:54.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376315+0000", 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9580000000000001e-06 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376320+0000", 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9320000000000001e-06 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376326+0000", 2024-09-12T07:19:54.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0390000000000004e-06 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11373 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377040+0000", 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018052763999999999, 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.021281472999999999, 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.611 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11373 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377040+0000", 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377040+0000", 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377040+0000", 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8400000000000004e-07 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377043+0000", 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2409999999999998e-06 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377049+0000", 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9500000000000004e-06 2024-09-12T07:19:54.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377054+0000", 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8749999999999999e-06 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11405 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383431+0000", 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011661306999999999, 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014941245000000001, 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.614 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11405 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383431+0000", 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383431+0000", 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383432+0000", 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5600000000000001e-07 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383439+0000", 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2289999999999997e-06 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383446+0000", 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8220000000000001e-06 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383452+0000", 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6610000000000003e-06 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11437 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.617 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390023+0000", 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0050689439999999997, 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0084010449999999993, 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11437 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390023+0000", 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390023+0000", 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390024+0000", 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4700000000000002e-07 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390026+0000", 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3240000000000001e-06 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390033+0000", 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.657e-06 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390038+0000", 2024-09-12T07:19:54.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.933e-06 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11469 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390789+0000", 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0043030710000000003, 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0076854779999999999, 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.621 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11469 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390789+0000", 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390789+0000", 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390790+0000", 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8299999999999998e-07 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390792+0000", 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9080000000000002e-06 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390797+0000", 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.75e-06 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390803+0000", 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.215e-06 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11310 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.369959+0000", 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025133256, 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028571861, 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.625 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.625 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.625 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.625 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11310 2024-09-12T07:19:54.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.625 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369959+0000", 2024-09-12T07:19:54.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369959+0000", 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369960+0000", 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369966+0000", 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6080000000000001e-06 2024-09-12T07:19:54.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369973+0000", 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7209999999999997e-06 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.369978+0000", 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9899999999999997e-06 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.631 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11342 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.631 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376332+0000", 2024-09-12T07:19:54.631 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018760652999999999, 2024-09-12T07:19:54.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022249367999999999, 2024-09-12T07:19:54.631 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.631 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.631 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.631 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.631 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.631 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11342 2024-09-12T07:19:54.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376332+0000", 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376332+0000", 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376332+0000", 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5699999999999998e-07 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376334+0000", 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.013e-06 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376340+0000", 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7980000000000002e-06 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376346+0000", 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1279999999999996e-06 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11374 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377062+0000", 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018030885999999999, 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.021573390000000001, 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.634 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11374 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377062+0000", 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377062+0000", 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377062+0000", 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.89e-07 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377064+0000", 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3010000000000001e-06 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377070+0000", 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9920000000000004e-06 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377075+0000", 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0969999999999996e-06 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11406 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383461+0000", 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011630959, 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015223108000000001, 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11406 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383461+0000", 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383461+0000", 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383462+0000", 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9800000000000004e-07 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383464+0000", 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383474+0000", 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7249999999999992e-06 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383481+0000", 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4200000000000001e-06 2024-09-12T07:19:54.640 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11438 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390045+0000", 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0050473139999999998, 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0087780099999999993, 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.641 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11438 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390045+0000", 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390045+0000", 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390046+0000", 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.32e-07 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390048+0000", 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1100000000000001e-06 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390054+0000", 2024-09-12T07:19:54.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5050000000000004e-06 2024-09-12T07:19:54.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390059+0000", 2024-09-12T07:19:54.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2349999999999999e-06 2024-09-12T07:19:54.644 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.644 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.644 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.644 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11470 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.644 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390810+0000", 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0042826599999999998, 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0080701809999999992, 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11470 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390810+0000", 2024-09-12T07:19:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390810+0000", 2024-09-12T07:19:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390810+0000", 2024-09-12T07:19:54.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390812+0000", 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1229999999999998e-06 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390819+0000", 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2570000000000004e-06 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390823+0000", 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7729999999999999e-06 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11311 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.370000+0000", 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025092650000000001, 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028935819000000002, 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11311 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370000+0000", 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.649 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370000+0000", 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370000+0000", 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3600000000000004e-07 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370003+0000", 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5689999999999999e-06 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370010+0000", 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1269999999999997e-06 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370015+0000", 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.366e-06 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11343 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376352+0000", 2024-09-12T07:19:54.651 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018740013999999999, 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022633288000000001, 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11343 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376352+0000", 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376352+0000", 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376353+0000", 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.03e-07 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376355+0000", 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0549999999999998e-06 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376361+0000", 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8850000000000004e-06 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376366+0000", 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.989e-06 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11375 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377082+0000", 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018010401999999998, 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.021956015999999998, 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11375 2024-09-12T07:19:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377082+0000", 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377082+0000", 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377082+0000", 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7300000000000002e-07 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377084+0000", 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0339999999999999e-06 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377091+0000", 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0759999999999999e-06 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377095+0000", 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9230000000000001e-06 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11407 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383491+0000", 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011601833000000001, 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015601703999999999, 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.658 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11407 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383491+0000", 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383491+0000", 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383491+0000", 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4499999999999997e-07 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383493+0000", 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3149999999999999e-06 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383506+0000", 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2736e-05 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383516+0000", 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4210000000000001e-06 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11439 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390083+0000", 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0050089979999999997, 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0090590900000000005, 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11439 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390083+0000", 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390083+0000", 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390084+0000", 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1600000000000001e-07 2024-09-12T07:19:54.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390087+0000", 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5170000000000002e-06 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390094+0000", 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4220000000000003e-06 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390100+0000", 2024-09-12T07:19:54.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6470000000000003e-06 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11471 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390832+0000", 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0042608580000000002, 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083681889999999998, 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.665 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11471 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390832+0000", 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390832+0000", 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390832+0000", 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5300000000000004e-07 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390834+0000", 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1799999999999999e-06 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390841+0000", 2024-09-12T07:19:54.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8650000000000003e-06 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390846+0000", 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.271e-06 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11312 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.370023+0000", 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025069786, 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029230138999999999, 2024-09-12T07:19:54.668 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11312 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370023+0000", 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370023+0000", 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370023+0000", 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370025+0000", 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1629999999999999e-06 2024-09-12T07:19:54.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370032+0000", 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1700000000000002e-06 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370036+0000", 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9359999999999998e-06 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11344 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376372+0000", 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018720516999999999, 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022933615000000001, 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11344 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.672 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376372+0000", 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376372+0000", 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376372+0000", 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8500000000000002e-07 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376374+0000", 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1270000000000001e-06 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376381+0000", 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1340000000000001e-06 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376386+0000", 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.118e-06 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11376 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377102+0000", 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017990784999999999, 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022254907000000001, 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11376 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377102+0000", 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377102+0000", 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377102+0000", 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3599999999999999e-07 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377104+0000", 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8789999999999999e-06 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377110+0000", 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1099999999999999e-06 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377115+0000", 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9849999999999997e-06 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11408 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383524+0000", 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011568439999999999, 2024-09-12T07:19:54.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015884935999999999, 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11408 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383524+0000", 2024-09-12T07:19:54.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383524+0000", 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383524+0000", 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6199999999999998e-07 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383527+0000", 2024-09-12T07:19:54.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.43e-06 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383535+0000", 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7190000000000001e-06 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383540+0000", 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4689999999999996e-06 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.681 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11440 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390107+0000", 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0049851909999999999, 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0093550300000000003, 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11440 2024-09-12T07:19:54.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390107+0000", 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390107+0000", 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390108+0000", 2024-09-12T07:19:54.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.63e-07 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390110+0000", 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3700000000000002e-06 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390116+0000", 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390124+0000", 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1099999999999997e-06 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11472 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390853+0000", 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0042396159999999999, 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0086619709999999992, 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.685 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11472 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390853+0000", 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390853+0000", 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390853+0000", 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7899999999999999e-07 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390855+0000", 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0549999999999998e-06 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390861+0000", 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9089999999999997e-06 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390872+0000", 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0715e-05 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11313 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.370043+0000", 2024-09-12T07:19:54.688 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025049673000000001, 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029525108000000001, 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11313 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370043+0000", 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370043+0000", 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370043+0000", 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9099999999999999e-07 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370045+0000", 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.136e-06 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370052+0000", 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2720000000000003e-06 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370057+0000", 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0560000000000004e-06 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.691 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11345 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376404+0000", 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018688046, 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023216311999999999, 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.692 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11345 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376404+0000", 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376404+0000", 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376405+0000", 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1900000000000002e-07 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376408+0000", 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6079999999999998e-06 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376416+0000", 2024-09-12T07:19:54.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1759999999999993e-06 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376422+0000", 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7370000000000003e-06 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11377 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377121+0000", 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017971009, 2024-09-12T07:19:54.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022550216000000001, 2024-09-12T07:19:54.696 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.696 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.696 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.696 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.696 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11377 2024-09-12T07:19:54.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.696 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377121+0000", 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377121+0000", 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377122+0000", 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377124+0000", 2024-09-12T07:19:54.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.311e-06 2024-09-12T07:19:54.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377130+0000", 2024-09-12T07:19:54.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0430000000000004e-06 2024-09-12T07:19:54.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377135+0000", 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7530000000000001e-06 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11409 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383558+0000", 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011534823, 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016168332000000001, 2024-09-12T07:19:54.699 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11409 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383558+0000", 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383558+0000", 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383558+0000", 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-07 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383560+0000", 2024-09-12T07:19:54.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8870000000000001e-06 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383564+0000", 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5190000000000003e-06 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383568+0000", 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1599999999999998e-06 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11441 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390132+0000", 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0049608580000000003, 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096459260000000008, 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11441 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.703 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390132+0000", 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390132+0000", 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390132+0000", 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4900000000000001e-07 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390139+0000", 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2910000000000002e-06 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390147+0000", 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8290000000000008e-06 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390152+0000", 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3940000000000004e-06 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11473 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390880+0000", 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.004212553, 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0089496650000000007, 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11473 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390880+0000", 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390880+0000", 2024-09-12T07:19:54.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390880+0000", 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390883+0000", 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6510000000000001e-06 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390889+0000", 2024-09-12T07:19:54.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1659999999999999e-06 2024-09-12T07:19:54.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390895+0000", 2024-09-12T07:19:54.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4310000000000002e-06 2024-09-12T07:19:54.709 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.709 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.709 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.709 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11314 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.370063+0000", 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025029172999999998, 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029823494999999998, 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11314 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370063+0000", 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370063+0000", 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370064+0000", 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370066+0000", 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8840000000000001e-06 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370072+0000", 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0650000000000004e-06 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370076+0000", 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6539999999999998e-06 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.712 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11346 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376429+0000", 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018663397000000002, 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023512313, 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11346 2024-09-12T07:19:54.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.714 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376429+0000", 2024-09-12T07:19:54.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376429+0000", 2024-09-12T07:19:54.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376429+0000", 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376437+0000", 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3030000000000002e-06 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376443+0000", 2024-09-12T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7360000000000004e-06 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376449+0000", 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.169e-06 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11378 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377142+0000", 2024-09-12T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017950747999999999, 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022850547999999998, 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11378 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377142+0000", 2024-09-12T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377142+0000", 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377142+0000", 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377144+0000", 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9580000000000001e-06 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377150+0000", 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0909999999999998e-06 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377156+0000", 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0549999999999996e-06 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11410 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383572+0000", 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011520417999999999, 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016469926999999999, 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11410 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383572+0000", 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383572+0000", 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383572+0000", 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0699999999999998e-07 2024-09-12T07:19:54.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383573+0000", 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.102e-06 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383579+0000", 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3680000000000001e-06 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383582+0000", 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9019999999999999e-06 2024-09-12T07:19:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11442 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390160+0000", 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0049325020000000001, 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0099344970000000005, 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11442 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390160+0000", 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390160+0000", 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390160+0000", 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4400000000000001e-07 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390162+0000", 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0600000000000002e-06 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390169+0000", 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.224e-06 2024-09-12T07:19:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390178+0000", 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3239999999999992e-06 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11474 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390901+0000", 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0041912119999999997, 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.009243345, 2024-09-12T07:19:54.726 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11474 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390901+0000", 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390901+0000", 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390902+0000", 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390904+0000", 2024-09-12T07:19:54.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4839999999999998e-06 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390910+0000", 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0020000000000003e-06 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390915+0000", 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6090000000000003e-06 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11315 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.370083+0000", 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025009706999999999, 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030117018999999998, 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11315 2024-09-12T07:19:54.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370083+0000", 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370083+0000", 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370083+0000", 2024-09-12T07:19:54.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9400000000000001e-07 2024-09-12T07:19:54.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370085+0000", 2024-09-12T07:19:54.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0449999999999999e-06 2024-09-12T07:19:54.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370091+0000", 2024-09-12T07:19:54.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0059999999999998e-06 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.370096+0000", 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8389999999999997e-06 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11347 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376456+0000", 2024-09-12T07:19:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018636755000000001, 2024-09-12T07:19:54.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023794190999999999, 2024-09-12T07:19:54.734 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.734 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.734 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.734 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.734 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.734 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11347 2024-09-12T07:19:54.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.734 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376456+0000", 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376456+0000", 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376456+0000", 2024-09-12T07:19:54.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376458+0000", 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9489999999999999e-06 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376464+0000", 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3060000000000003e-06 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376469+0000", 2024-09-12T07:19:54.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0250000000000002e-06 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11379 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377163+0000", 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017929301000000002, 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023139862000000001, 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11379 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377163+0000", 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377163+0000", 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377164+0000", 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.353e-06 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377167+0000", 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.04e-06 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377174+0000", 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1389999999999998e-06 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377179+0000", 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.417e-06 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11411 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383585+0000", 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011507145, 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016776368, 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.740 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11411 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383585+0000", 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383585+0000", 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383586+0000", 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6199999999999999e-07 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383587+0000", 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1230000000000001e-06 2024-09-12T07:19:54.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383590+0000", 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0809999999999998e-06 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383592+0000", 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6170000000000001e-06 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11443 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390192+0000", 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0049000629999999996, 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010221729000000001, 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11443 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390192+0000", 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390192+0000", 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390193+0000", 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5499999999999998e-07 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390195+0000", 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1710000000000001e-06 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390202+0000", 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2429999999999999e-06 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390208+0000", 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4909999999999996e-06 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.746 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11475 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390921+0000", 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0041712290000000003, 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0095432399999999997, 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11475 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.747 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390921+0000", 2024-09-12T07:19:54.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390921+0000", 2024-09-12T07:19:54.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390922+0000", 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1199999999999998e-07 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390924+0000", 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.074e-06 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390930+0000", 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9830000000000001e-06 2024-09-12T07:19:54.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390935+0000", 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-06 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11348 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376476+0000", 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018616704000000001, 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024042483999999999, 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.750 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11348 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376476+0000", 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376476+0000", 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376476+0000", 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9999999999999998e-07 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376478+0000", 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0710000000000002e-06 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376484+0000", 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9379999999999997e-06 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376489+0000", 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9459999999999997e-06 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.753 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11380 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377187+0000", 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017905047, 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023379332999999999, 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11380 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377187+0000", 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377187+0000", 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377188+0000", 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9500000000000003e-07 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377190+0000", 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.119e-06 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377196+0000", 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9739999999999999e-06 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377201+0000", 2024-09-12T07:19:54.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8049999999999997e-06 2024-09-12T07:19:54.757 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.757 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.757 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.757 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11412 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.757 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383596+0000", 2024-09-12T07:19:54.757 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011496895, 2024-09-12T07:19:54.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017023982999999999, 2024-09-12T07:19:54.757 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11412 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383596+0000", 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383596+0000", 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383596+0000", 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3200000000000001e-07 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383597+0000", 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.034e-06 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383600+0000", 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9579999999999999e-06 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383602+0000", 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.712e-06 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11444 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.760 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390215+0000", 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0048773059999999997, 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010454237999999999, 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11444 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390215+0000", 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390215+0000", 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390216+0000", 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390218+0000", 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2699999999999999e-06 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390224+0000", 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1410000000000002e-06 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390229+0000", 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1710000000000001e-06 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11476 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390943+0000", 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.004149481, 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0097782979999999995, 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.764 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11476 2024-09-12T07:19:54.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.765 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390943+0000", 2024-09-12T07:19:54.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390943+0000", 2024-09-12T07:19:54.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390944+0000", 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6400000000000002e-07 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390946+0000", 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0470000000000001e-06 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390951+0000", 2024-09-12T07:19:54.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9200000000000001e-06 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390957+0000", 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6790000000000002e-06 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11349 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376504+0000", 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018587966000000001, 2024-09-12T07:19:54.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024268912, 2024-09-12T07:19:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11349 2024-09-12T07:19:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376504+0000", 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376504+0000", 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376505+0000", 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2699999999999999e-07 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376507+0000", 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2579999999999998e-06 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376514+0000", 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8820000000000003e-06 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376519+0000", 2024-09-12T07:19:54.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1490000000000002e-06 2024-09-12T07:19:54.771 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.771 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.771 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.771 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11381 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.771 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377208+0000", 2024-09-12T07:19:54.771 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017884923, 2024-09-12T07:19:54.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023619857000000001, 2024-09-12T07:19:54.771 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.772 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.772 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.772 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.772 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.772 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.772 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11381 2024-09-12T07:19:54.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.772 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377208+0000", 2024-09-12T07:19:54.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377208+0000", 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377208+0000", 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377214+0000", 2024-09-12T07:19:54.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:19:54.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377221+0000", 2024-09-12T07:19:54.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.387e-06 2024-09-12T07:19:54.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377226+0000", 2024-09-12T07:19:54.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.977e-06 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11413 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383606+0000", 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011486788, 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017277435000000001, 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.775 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11413 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383606+0000", 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383606+0000", 2024-09-12T07:19:54.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383606+0000", 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.29e-07 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383607+0000", 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0559999999999999e-06 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383610+0000", 2024-09-12T07:19:54.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9000000000000002e-06 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383612+0000", 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.509e-06 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11445 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390236+0000", 2024-09-12T07:19:54.778 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0048565860000000004, 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010697585000000001, 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11445 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390236+0000", 2024-09-12T07:19:54.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390236+0000", 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390236+0000", 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.65e-07 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390238+0000", 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9659999999999999e-06 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390244+0000", 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.117e-06 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390249+0000", 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.028e-06 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11477 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390964+0000", 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0041288000000000002, 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010023031, 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11477 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390964+0000", 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390964+0000", 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390964+0000", 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8699999999999995e-07 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390971+0000", 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.759e-06 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390977+0000", 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4010000000000002e-06 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390982+0000", 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1200000000000001e-06 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11350 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376527+0000", 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018565874, 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024511532999999999, 2024-09-12T07:19:54.785 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11350 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376527+0000", 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376527+0000", 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376527+0000", 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376529+0000", 2024-09-12T07:19:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0710000000000002e-06 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376535+0000", 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3199999999999996e-06 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376541+0000", 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0629999999999996e-06 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11382 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377244+0000", 2024-09-12T07:19:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017848813000000002, 2024-09-12T07:19:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023847258, 2024-09-12T07:19:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11382 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377244+0000", 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377244+0000", 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377244+0000", 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8400000000000003e-07 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377247+0000", 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4609999999999998e-06 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377254+0000", 2024-09-12T07:19:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2840000000000001e-06 2024-09-12T07:19:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377260+0000", 2024-09-12T07:19:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5759999999999996e-06 2024-09-12T07:19:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11414 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383616+0000", 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011476872000000001, 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017528065999999998, 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11414 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383616+0000", 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383616+0000", 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383616+0000", 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2700000000000001e-07 2024-09-12T07:19:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383617+0000", 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.47e-07 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383620+0000", 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8669999999999998e-06 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383622+0000", 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.588e-06 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11446 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390257+0000", 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0048357870000000002, 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01094036, 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11446 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390257+0000", 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390257+0000", 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390257+0000", 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6800000000000001e-07 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390259+0000", 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9889999999999999e-06 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390265+0000", 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.224e-06 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390272+0000", 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9110000000000004e-06 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11478 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390989+0000", 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0041030930000000004, 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010256634000000001, 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11478 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390989+0000", 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390989+0000", 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390990+0000", 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3500000000000002e-07 2024-09-12T07:19:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390992+0000", 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.12e-06 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390998+0000", 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.229e-06 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391003+0000", 2024-09-12T07:19:54.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.656e-06 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11351 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376549+0000", 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0185436, 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024752505000000001, 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.803 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11351 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376549+0000", 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376549+0000", 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376549+0000", 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4900000000000001e-07 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376552+0000", 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.373e-06 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376559+0000", 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4379999999999998e-06 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376564+0000", 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0939999999999998e-06 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11383 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377267+0000", 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017825374000000001, 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024083954000000001, 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11383 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377267+0000", 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377267+0000", 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377367+0000", 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377370+0000", 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2899999999999998e-06 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377379+0000", 2024-09-12T07:19:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9069999999999996e-06 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377385+0000", 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9699999999999996e-06 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11415 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383625+0000", 2024-09-12T07:19:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011467112, 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017783106, 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11415 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383625+0000", 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383625+0000", 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383626+0000", 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6300000000000001e-07 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383627+0000", 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0470000000000001e-06 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383630+0000", 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9620000000000002e-06 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383632+0000", 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4490000000000002e-06 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11447 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390291+0000", 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0048010090000000002, 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011167451, 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11447 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390291+0000", 2024-09-12T07:19:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390291+0000", 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390292+0000", 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5300000000000004e-07 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390294+0000", 2024-09-12T07:19:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2979999999999999e-06 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390302+0000", 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4610000000000002e-06 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390307+0000", 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5559999999999998e-06 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11479 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391009+0000", 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0040829450000000002, 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010502281, 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11479 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391009+0000", 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391009+0000", 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391010+0000", 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5400000000000002e-07 2024-09-12T07:19:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391012+0000", 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1880000000000001e-06 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391019+0000", 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1690000000000003e-06 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391024+0000", 2024-09-12T07:19:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9529999999999998e-06 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11352 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376571+0000", 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018521579, 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024992874000000002, 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11352 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376571+0000", 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376571+0000", 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376571+0000", 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376573+0000", 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0389999999999999e-06 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376579+0000", 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.054e-06 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376584+0000", 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.887e-06 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11384 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.377397+0000", 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017695081000000001, 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024219747, 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11384 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377397+0000", 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.377397+0000", 2024-09-12T07:19:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382917+0000", 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055199309999999996 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382922+0000", 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1599999999999997e-06 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382936+0000", 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3548e-05 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382944+0000", 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.892e-06 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11416 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383635+0000", 2024-09-12T07:19:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011457073999999999, 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018030912999999999, 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11416 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383635+0000", 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383635+0000", 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383636+0000", 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2000000000000001e-07 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383637+0000", 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8899999999999998e-07 2024-09-12T07:19:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383639+0000", 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.886e-06 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383642+0000", 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3889999999999999e-06 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11448 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390315+0000", 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0047775550000000002, 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011400914, 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11448 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390315+0000", 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390315+0000", 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390315+0000", 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390317+0000", 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.013e-06 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390323+0000", 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9669999999999998e-06 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390329+0000", 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5040000000000002e-06 2024-09-12T07:19:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11480 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391030+0000", 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0040620389999999999, 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010738091999999999, 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11480 2024-09-12T07:19:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391030+0000", 2024-09-12T07:19:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391030+0000", 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391031+0000", 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.08e-07 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391033+0000", 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0669999999999999e-06 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391039+0000", 2024-09-12T07:19:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6030000000000001e-06 2024-09-12T07:19:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391045+0000", 2024-09-12T07:19:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.186e-06 2024-09-12T07:19:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11353 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376591+0000", 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018501725, 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025229705000000002, 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11353 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376591+0000", 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376591+0000", 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376591+0000", 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6600000000000002e-07 2024-09-12T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376593+0000", 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.018e-06 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376599+0000", 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0630000000000002e-06 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376604+0000", 2024-09-12T07:19:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0200000000000002e-06 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11385 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.382957+0000", 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.012135633, 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018923481999999998, 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11385 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382957+0000", 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382957+0000", 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382957+0000", 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382960+0000", 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7420000000000002e-06 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382967+0000", 2024-09-12T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6599999999999998e-06 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382976+0000", 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8410000000000006e-06 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11417 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383647+0000", 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011445433, 2024-09-12T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018284426999999999, 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11417 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383647+0000", 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383647+0000", 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383648+0000", 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383659+0000", 2024-09-12T07:19:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1732e-05 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383666+0000", 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5459999999999997e-06 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383671+0000", 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5049999999999998e-06 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11449 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390336+0000", 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0047565760000000002, 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011647128, 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11449 2024-09-12T07:19:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390336+0000", 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390336+0000", 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390336+0000", 2024-09-12T07:19:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9999999999999998e-07 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390338+0000", 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1780000000000002e-06 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390345+0000", 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1090000000000003e-06 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390351+0000", 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.981e-06 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11481 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391061+0000", 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0040316789999999998, 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010971516000000001, 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11481 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391061+0000", 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391061+0000", 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391061+0000", 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4799999999999998e-07 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391064+0000", 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3350000000000001e-06 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391071+0000", 2024-09-12T07:19:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9949999999999999e-06 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391076+0000", 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4360000000000001e-06 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11354 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376611+0000", 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018481775999999998, 2024-09-12T07:19:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025478282000000001, 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11354 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376611+0000", 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376611+0000", 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376611+0000", 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.39e-07 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376613+0000", 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1349999999999999e-06 2024-09-12T07:19:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376619+0000", 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2029999999999997e-06 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376624+0000", 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8130000000000003e-06 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11386 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.382987+0000", 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.012105941, 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019151945, 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11386 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382987+0000", 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382987+0000", 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382987+0000", 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0200000000000002e-07 2024-09-12T07:19:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382989+0000", 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4660000000000002e-06 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.382997+0000", 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4420000000000001e-06 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383009+0000", 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2357999999999999e-05 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11418 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383682+0000", 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.01141063, 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018511316999999999, 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11418 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383682+0000", 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383682+0000", 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383682+0000", 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383685+0000", 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2989999999999999e-06 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383692+0000", 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.593e-06 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383697+0000", 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9599999999999999e-06 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11450 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390358+0000", 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0047343690000000004, 2024-09-12T07:19:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011884778, 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11450 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390358+0000", 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390358+0000", 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390358+0000", 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.08e-07 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390361+0000", 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1289999999999999e-06 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390367+0000", 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390372+0000", 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1909999999999999e-06 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11482 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391083+0000", 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0040089540000000003, 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011213742000000001, 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11482 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391083+0000", 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391083+0000", 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391084+0000", 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391086+0000", 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2259999999999999e-06 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391092+0000", 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2330000000000003e-06 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391097+0000", 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9010000000000002e-06 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11355 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376632+0000", 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018460689999999998, 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025717943, 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11355 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376632+0000", 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376632+0000", 2024-09-12T07:19:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376632+0000", 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4900000000000001e-07 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376634+0000", 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0679999999999999e-06 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376640+0000", 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0630000000000002e-06 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376645+0000", 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.989e-06 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11387 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383018+0000", 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.012074474, 2024-09-12T07:19:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019531195000000001, 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11387 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383018+0000", 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383018+0000", 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383018+0000", 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8299999999999997e-07 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383021+0000", 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2369999999999999e-06 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383029+0000", 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3170000000000002e-06 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383040+0000", 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0524e-05 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11419 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383705+0000", 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011387771, 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01889596, 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11419 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383705+0000", 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383705+0000", 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383705+0000", 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.58e-07 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383707+0000", 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0389999999999999e-06 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383714+0000", 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3489999999999997e-06 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383718+0000", 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7690000000000004e-06 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11451 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390378+0000", 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0047144129999999998, 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012276499999999999, 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11451 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390378+0000", 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390378+0000", 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390378+0000", 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6600000000000002e-07 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390380+0000", 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9549999999999999e-06 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390388+0000", 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9680000000000006e-06 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390393+0000", 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9949999999999996e-06 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11483 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391104+0000", 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0039885950000000002, 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011599625000000001, 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11483 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.882 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391104+0000", 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391104+0000", 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391104+0000", 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0999999999999999e-07 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.883 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391106+0000", 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0899999999999999e-06 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391114+0000", 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2389999999999996e-06 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391118+0000", 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8840000000000002e-06 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.884 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11356 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376652+0000", 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018440733000000001, 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026103309000000002, 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.885 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11356 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376652+0000", 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376652+0000", 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376652+0000", 2024-09-12T07:19:54.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5999999999999999e-07 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376654+0000", 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9709999999999998e-06 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376660+0000", 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6829999999999996e-06 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376665+0000", 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9490000000000004e-06 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11388 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383047+0000", 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.012045169, 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019761102999999999, 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.888 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11388 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383047+0000", 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383047+0000", 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383048+0000", 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2499999999999995e-07 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383050+0000", 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.289e-06 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383059+0000", 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0270000000000001e-06 2024-09-12T07:19:54.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383065+0000", 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.609e-06 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11420 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383725+0000", 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011367463, 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019133142999999998, 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.891 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11420 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383725+0000", 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.892 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383725+0000", 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383725+0000", 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383727+0000", 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.928e-06 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383733+0000", 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1310000000000003e-06 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383740+0000", 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3679999999999998e-06 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.894 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11452 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390409+0000", 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0046831210000000002, 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01250655, 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11452 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.895 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390409+0000", 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390409+0000", 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390410+0000", 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7299999999999996e-07 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390412+0000", 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5359999999999999e-06 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390420+0000", 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8830000000000006e-06 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390426+0000", 2024-09-12T07:19:54.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6550000000000001e-06 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11484 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391125+0000", 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0039673479999999999, 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011840293, 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.898 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11484 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391125+0000", 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391125+0000", 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391125+0000", 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6300000000000001e-07 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391128+0000", 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1969999999999999e-06 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391134+0000", 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.976e-06 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391139+0000", 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0479999999999998e-06 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11357 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376671+0000", 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018421314000000001, 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026349434000000001, 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.901 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11357 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376671+0000", 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376671+0000", 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376671+0000", 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6100000000000002e-07 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376674+0000", 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0549999999999998e-06 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376679+0000", 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9290000000000003e-06 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376684+0000", 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8160000000000001e-06 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11389 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383073+0000", 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.012019594999999999, 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019997212, 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.904 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11389 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383073+0000", 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383073+0000", 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383073+0000", 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.63e-07 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383076+0000", 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2309999999999999e-06 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383084+0000", 2024-09-12T07:19:54.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4169999999999992e-06 2024-09-12T07:19:54.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383089+0000", 2024-09-12T07:19:54.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0540000000000002e-06 2024-09-12T07:19:54.907 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.907 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.907 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11421 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383746+0000", 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011346106, 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019378557000000001, 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11421 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383746+0000", 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383746+0000", 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383747+0000", 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.08e-07 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383749+0000", 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.903e-06 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383755+0000", 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1840000000000004e-06 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383761+0000", 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6729999999999997e-06 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.910 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11453 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390433+0000", 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0046590609999999999, 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0127405, 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11453 2024-09-12T07:19:54.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390433+0000", 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390433+0000", 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390434+0000", 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2699999999999999e-07 2024-09-12T07:19:54.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390441+0000", 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9240000000000001e-06 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390448+0000", 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7750000000000004e-06 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390453+0000", 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4160000000000003e-06 2024-09-12T07:19:54.913 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11485 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391145+0000", 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0039475229999999997, 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012081802000000001, 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.914 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11485 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391145+0000", 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391145+0000", 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.915 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391146+0000", 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8800000000000002e-07 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391148+0000", 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.018e-06 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391153+0000", 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7300000000000002e-06 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.916 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391158+0000", 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8940000000000001e-06 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11358 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376690+0000", 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018402182, 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026587598, 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.917 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11358 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376690+0000", 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376690+0000", 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376691+0000", 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6199999999999999e-07 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376702+0000", 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1452e-05 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.919 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376709+0000", 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.046e-06 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376715+0000", 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6919999999999999e-06 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11390 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.920 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383098+0000", 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011994579999999999, 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020233260999999999, 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11390 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383098+0000", 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383098+0000", 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383098+0000", 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383101+0000", 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1500000000000002e-06 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383107+0000", 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3450000000000002e-06 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383119+0000", 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1955e-05 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.923 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11422 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383771+0000", 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011321233, 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019611119999999999, 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11422 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383771+0000", 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383771+0000", 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383772+0000", 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8099999999999998e-07 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383774+0000", 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4849999999999999e-06 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383781+0000", 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5420000000000002e-06 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383788+0000", 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6990000000000003e-06 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11454 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390460+0000", 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0046321380000000001, 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012975321, 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.927 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11454 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390460+0000", 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390460+0000", 2024-09-12T07:19:54.928 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390461+0000", 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4299999999999998e-07 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390463+0000", 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0700000000000001e-06 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.929 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390469+0000", 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9970000000000004e-06 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390474+0000", 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2669999999999997e-06 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11486 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391165+0000", 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0039277569999999996, 2024-09-12T07:19:54.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012322336, 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11486 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391165+0000", 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391165+0000", 2024-09-12T07:19:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391165+0000", 2024-09-12T07:19:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6399999999999998e-07 2024-09-12T07:19:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.932 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391167+0000", 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0549999999999998e-06 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391173+0000", 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1500000000000004e-06 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391178+0000", 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7110000000000003e-06 2024-09-12T07:19:54.933 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11359 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376723+0000", 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018369814000000002, 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026818695, 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.934 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11359 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376723+0000", 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376723+0000", 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376723+0000", 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2399999999999998e-07 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376725+0000", 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.086e-06 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376731+0000", 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0159999999999997e-06 2024-09-12T07:19:54.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376736+0000", 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11391 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383127+0000", 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011965724, 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020464263, 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.937 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11391 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383127+0000", 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.938 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383127+0000", 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383127+0000", 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7899999999999999e-07 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383130+0000", 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3640000000000002e-06 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383138+0000", 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4749999999999993e-06 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383147+0000", 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5339999999999999e-06 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11423 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.940 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383796+0000", 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011296275, 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019843942, 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11423 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383796+0000", 2024-09-12T07:19:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383796+0000", 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383797+0000", 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.942 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383799+0000", 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1380000000000001e-06 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383805+0000", 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4150000000000004e-06 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383810+0000", 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0749999999999997e-06 2024-09-12T07:19:54.943 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11455 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390480+0000", 2024-09-12T07:19:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0046119910000000002, 2024-09-12T07:19:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01321222, 2024-09-12T07:19:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.944 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11455 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390480+0000", 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390480+0000", 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390481+0000", 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390483+0000", 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.244e-06 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.946 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390489+0000", 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1199999999999999e-06 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390494+0000", 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0669999999999999e-06 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11487 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.947 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391184+0000", 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0039081150000000002, 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012557820000000001, 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11487 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391184+0000", 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391184+0000", 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391185+0000", 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6600000000000002e-07 2024-09-12T07:19:54.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391187+0000", 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0099999999999998e-06 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391193+0000", 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8969999999999996e-06 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391198+0000", 2024-09-12T07:19:54.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8350000000000003e-06 2024-09-12T07:19:54.951 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.951 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.951 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.951 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11360 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.951 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376743+0000", 2024-09-12T07:19:54.951 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018349308000000002, 2024-09-12T07:19:54.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027053526000000001, 2024-09-12T07:19:54.951 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11360 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376743+0000", 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376743+0000", 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376744+0000", 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8700000000000001e-07 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376746+0000", 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0370000000000001e-06 2024-09-12T07:19:54.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376753+0000", 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376759+0000", 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3059999999999997e-06 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11392 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383155+0000", 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011937691, 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020690101999999998, 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11392 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.955 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383155+0000", 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383155+0000", 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383155+0000", 2024-09-12T07:19:54.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4999999999999998e-07 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383157+0000", 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.215e-06 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383167+0000", 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4890000000000001e-06 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383174+0000", 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0600000000000002e-06 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11424 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383820+0000", 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011272575, 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020084541000000001, 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11424 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383820+0000", 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383820+0000", 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383820+0000", 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8100000000000003e-07 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383823+0000", 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1380000000000001e-06 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383831+0000", 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7180000000000002e-06 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383838+0000", 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.99e-06 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11456 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390503+0000", 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0045899360000000002, 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013452047999999999, 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11456 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390503+0000", 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390503+0000", 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390503+0000", 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-07 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390505+0000", 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1789999999999998e-06 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390513+0000", 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2989999999999999e-06 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390518+0000", 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7729999999999996e-06 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11488 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391204+0000", 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.003888828, 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012805473, 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11488 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391204+0000", 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391204+0000", 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391204+0000", 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8099999999999998e-07 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391206+0000", 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9599999999999999e-06 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391212+0000", 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1140000000000002e-06 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391217+0000", 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2639999999999999e-06 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.967 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11361 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376767+0000", 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018325226, 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027292984999999999, 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11361 2024-09-12T07:19:54.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376767+0000", 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376767+0000", 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376768+0000", 2024-09-12T07:19:54.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1100000000000001e-07 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376770+0000", 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0439999999999998e-06 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376776+0000", 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0889999999999996e-06 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376780+0000", 2024-09-12T07:19:54.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6319999999999999e-06 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11393 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383198+0000", 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011894811, 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020916262000000001, 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.971 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11393 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383198+0000", 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383198+0000", 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383198+0000", 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.84e-07 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383204+0000", 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3600000000000001e-06 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383211+0000", 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5220000000000004e-06 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383215+0000", 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1570000000000002e-06 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11425 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383857+0000", 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011235113, 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020307248999999999, 2024-09-12T07:19:54.974 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11425 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383857+0000", 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383857+0000", 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383858+0000", 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383860+0000", 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3259999999999998e-06 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383872+0000", 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1595e-05 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383877+0000", 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5999999999999997e-06 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11457 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.977 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390525+0000", 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0045672159999999998, 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013691715, 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11457 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390525+0000", 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390525+0000", 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390526+0000", 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.75e-07 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390528+0000", 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.956e-06 2024-09-12T07:19:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390534+0000", 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3450000000000002e-06 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390539+0000", 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8369999999999996e-06 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.980 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11489 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.981 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391232+0000", 2024-09-12T07:19:54.981 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0038601550000000001, 2024-09-12T07:19:54.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013034281, 2024-09-12T07:19:54.985 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.985 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.985 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.985 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.985 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.985 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.985 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11489 2024-09-12T07:19:54.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.985 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391232+0000", 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391232+0000", 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391233+0000", 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1099999999999996e-07 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391235+0000", 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5289999999999998e-06 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391242+0000", 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9609999999999999e-06 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391248+0000", 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.417e-06 2024-09-12T07:19:54.987 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11362 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376787+0000", 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018305498999999999, 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027533875999999999, 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.988 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11362 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376787+0000", 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376787+0000", 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376787+0000", 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.65e-07 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376790+0000", 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.221e-06 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376795+0000", 2024-09-12T07:19:54.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8819999999999998e-06 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376800+0000", 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8350000000000003e-06 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11394 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383220+0000", 2024-09-12T07:19:54.991 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011872156, 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.021149778000000001, 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11394 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383220+0000", 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383220+0000", 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383221+0000", 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-07 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383222+0000", 2024-09-12T07:19:54.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.311e-06 2024-09-12T07:19:54.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383225+0000", 2024-09-12T07:19:54.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3069999999999998e-06 2024-09-12T07:19:54.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383228+0000", 2024-09-12T07:19:54.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7930000000000002e-06 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11426 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383886+0000", 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011206504000000001, 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020538091000000001, 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11426 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383886+0000", 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383886+0000", 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383886+0000", 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8800000000000003e-07 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383889+0000", 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.458e-06 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383897+0000", 2024-09-12T07:19:54.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.259e-06 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383902+0000", 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1490000000000002e-06 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11458 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390546+0000", 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0045469229999999996, 2024-09-12T07:19:54.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013929228, 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11458 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390546+0000", 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:54.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:55.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390546+0000", 2024-09-12T07:19:55.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:55.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390546+0000", 2024-09-12T07:19:55.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.08e-07 2024-09-12T07:19:55.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390548+0000", 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3080000000000002e-06 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390554+0000", 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2169999999999999e-06 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390559+0000", 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8369999999999996e-06 2024-09-12T07:19:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.002 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:55.002 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.002 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11490 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:55.002 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391272+0000", 2024-09-12T07:19:55.002 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0038205940000000001, 2024-09-12T07:19:55.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013251924999999999, 2024-09-12T07:19:55.002 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:55.002 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:55.002 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:55.002 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11490 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391272+0000", 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391272+0000", 2024-09-12T07:19:55.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391272+0000", 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9599999999999999e-07 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391275+0000", 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6180000000000002e-06 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:55.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391283+0000", 2024-09-12T07:19:55.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0150000000000003e-06 2024-09-12T07:19:55.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:55.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391289+0000", 2024-09-12T07:19:55.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3609999999999997e-06 2024-09-12T07:19:55.005 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.005 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:55.005 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11363 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.376809+0000", 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018283832999999999, 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027774233999999998, 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11363 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:55.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:55.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376809+0000", 2024-09-12T07:19:55.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:55.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376809+0000", 2024-09-12T07:19:55.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:55.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376809+0000", 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5700000000000002e-07 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376811+0000", 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.911e-06 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376817+0000", 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9739999999999999e-06 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.376822+0000", 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0880000000000002e-06 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11395 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383232+0000", 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011860175000000001, 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.021401646999999999, 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:55.009 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11395 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383232+0000", 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383232+0000", 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383232+0000", 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.22e-07 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383234+0000", 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.128e-06 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383237+0000", 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0790000000000001e-06 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383239+0000", 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6749999999999998e-06 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11427 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.383916+0000", 2024-09-12T07:19:55.012 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011176347, 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020771648, 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11427 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383916+0000", 2024-09-12T07:19:55.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383916+0000", 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383916+0000", 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9999999999999999e-07 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383918+0000", 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6190000000000001e-06 2024-09-12T07:19:55.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383923+0000", 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4190000000000001e-06 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.383927+0000", 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7000000000000002e-06 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11459 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.390566+0000", 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0045265740000000002, 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014171885, 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11459 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390566+0000", 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390566+0000", 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390566+0000", 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9200000000000002e-07 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:55.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390568+0000", 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.978e-06 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390574+0000", 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9290000000000003e-06 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.390579+0000", 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:11491 3.1e 3.17dcfa5e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.391297+0000", 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.003795704, 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013493359, 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11491 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391297+0000", 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391297+0000", 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391297+0000", 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391300+0000", 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.322e-06 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391306+0000", 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3160000000000002e-06 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.391311+0000", 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2109999999999997e-06 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.021 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:55.022 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.022 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:55.022 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T07:19:55.022 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 180 2024-09-12T07:19:55.022 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:55.023 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:19:55.023 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:19:55.024 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:19:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:19:55.025 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:19:55.025 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:55.025+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:55.025 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:19:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:19:55.026 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:19:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:19:55.026 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:19:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:19:55.028 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-12T07:19:55.028 INFO:teuthology.orchestra.run.smithi046.stdout:690 punch from 0x60b4a4f to 0x60b51ba, (0x76b bytes) 2024-09-12T07:19:55.028 INFO:teuthology.orchestra.run.smithi046.stdout:691 trunc from 0xba24625 to 0x6bc936f 2024-09-12T07:19:55.028 INFO:teuthology.orchestra.run.smithi046.stdout:692 read 0x1341bf9 thru 0x1351954 (0xfd5c bytes) 2024-09-12T07:19:55.028 INFO:teuthology.orchestra.run.smithi046.stdout:693 read 0xc9d655 thru 0xcad100 (0xfaac bytes) 2024-09-12T07:19:55.028 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:55.029 INFO:teuthology.orchestra.run.smithi046.stdout:696 read 0xb0806e thru 0xb0886a (0x7fd bytes) 2024-09-12T07:19:55.029 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:55.029 INFO:teuthology.orchestra.run.smithi046.stdout:698 write 0xa878cf5 thru 0xa886eae (0xe1ba bytes) 2024-09-12T07:19:55.029 INFO:teuthology.orchestra.run.smithi046.stdout:699 write 0x1f7554d thru 0x1f84fec (0xfaa0 bytes) 2024-09-12T07:19:55.124 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:55.126+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:55.220 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:55.220 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:55.225 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:55.227+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:55.231 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:19:55.326 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:55.327+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:55.395 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:55.396 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:55.407 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:19:55.426 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:55.428+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:55.528 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:55.529+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:55.530 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:55.530 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:55.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:19:55.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T07:19:55.630 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:55.632+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:55.728 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:55.729 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:55.729 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:55.730+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:55.737 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:19:55.830 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:55.831+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:55.891 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:55.891 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:55.899 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:19:55.943 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:55.945+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:56.007 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:56.008 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:56.016 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:19:56.058 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:56.059+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is set 2024-09-12T07:19:56.132 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:56.133+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:56.169 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:56.169 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:56.174 INFO:teuthology.orchestra.run.smithi046.stdout:701 writesame 0x5ecfedd thru 0x5ed8810 data_size 0x6(0x8934 bytes) 2024-09-12T07:19:56.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:19:56.179 INFO:teuthology.orchestra.run.smithi046.stdout:705 write 0x5a2ea6f thru 0x5a39a15 (0xafa7 bytes) 2024-09-12T07:19:56.182 INFO:teuthology.orchestra.run.smithi046.stdout:706 read 0xc4797f thru 0xc4c02b (0x46ad bytes) 2024-09-12T07:19:56.188 INFO:teuthology.orchestra.run.smithi046.stdout:707 trunc from 0xa886eaf to 0x8542098 2024-09-12T07:19:56.196 INFO:teuthology.orchestra.run.smithi046.stdout:708 punch from 0x50d8620 to 0x50de508, (0x5ee8 bytes) 2024-09-12T07:19:56.196 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:56.197 INFO:teuthology.orchestra.run.smithi046.stdout:711 trunc from 0x8542098 to 0x6e216fc 2024-09-12T07:19:56.210 INFO:teuthology.orchestra.run.smithi046.stdout:713 trunc from 0x6e216fc to 0x6654d7d 2024-09-12T07:19:56.222 INFO:teuthology.orchestra.run.smithi046.stdout:714 read 0x220fbb2 thru 0x2211a46 (0x1e95 bytes) 2024-09-12T07:19:56.222 INFO:teuthology.orchestra.run.smithi046.stdout:truncating to largest ever: 0xeb6e6a3 2024-09-12T07:19:56.232 INFO:teuthology.orchestra.run.smithi046.stdout:716 trunc from 0x6654d7d to 0xeb6e6a3 2024-09-12T07:19:56.232 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:56.234+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:56.237 INFO:teuthology.orchestra.run.smithi046.stdout:717 punch from 0x12253d2 to 0x1230786, (0xb3b4 bytes) 2024-09-12T07:19:56.237 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:56.238 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:56.246 INFO:teuthology.orchestra.run.smithi046.stdout:722 trunc from 0xeb6e6a3 to 0x6d9c7a3 2024-09-12T07:19:56.255 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:56.256 INFO:teuthology.orchestra.run.smithi046.stdout:726 punch from 0x669fb8f to 0x66a2f32, (0x33a3 bytes) 2024-09-12T07:19:56.259 INFO:teuthology.orchestra.run.smithi046.stdout:731 trunc from 0x6d9c7a3 to 0x3e6846a 2024-09-12T07:19:56.321 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:56.321 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:56.332 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:19:56.334 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:56.335+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:56.435 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:56.437+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:56.508 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:56.508 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:56.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:19:56.561 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:56.563+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:56.637 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:56.638+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:56.665 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:56.666 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:56.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:19:56.738 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:56.739+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:56.839 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:56.840+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:56.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:19:56.939 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:56.941+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:57.003 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:57.003 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:57.012 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:19:57.028 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:19:57.057 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:57.058+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:57.141 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:57.143+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:57.165 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:19:57.165 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:19:57.179 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:19:57.214 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:19:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:19:57.233 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:19:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:19:57.242 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:57.244+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:57.253 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:19:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:19:57.270 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:19:57.270 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:19:57.301 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:19:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:19:57.315 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:57.315 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:19:57.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:19:57.316 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:19:57.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.316 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:12 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T07:19:57.316 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:44.467367+0000", 2024-09-12T07:19:57.316 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.847443618, 2024-09-12T07:19:57.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027275540000000001, 2024-09-12T07:19:57.316 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.316 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12 2024-09-12T07:19:57.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.317 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467367+0000", 2024-09-12T07:19:57.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467367+0000", 2024-09-12T07:19:57.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467368+0000", 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1519999999999999e-06 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467372+0000", 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1980000000000003e-06 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467380+0000", 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.515e-06 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467387+0000", 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0570000000000004e-06 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467437+0000", 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9738999999999999e-05 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.467447+0000", 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0227999999999999e-05 2024-09-12T07:19:57.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.468920+0000", 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014732059999999999 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469023+0000", 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103087 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469168+0000", 2024-09-12T07:19:57.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014458099999999999 2024-09-12T07:19:57.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.469640+0000", 2024-09-12T07:19:57.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047246499999999998 2024-09-12T07:19:57.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470041+0000", 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040072399999999998 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470072+0000", 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1080000000000001e-05 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:44.470095+0000", 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2521999999999999e-05 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.323 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.324 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:492 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [watch unwatch cookie 140539785655440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:57.324 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.555907+0000", 2024-09-12T07:19:57.324 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.7589032249999992, 2024-09-12T07:19:57.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029325409999999999, 2024-09-12T07:19:57.324 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 492 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555907+0000", 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555907+0000", 2024-09-12T07:19:57.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555908+0000", 2024-09-12T07:19:57.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4499999999999997e-07 2024-09-12T07:19:57.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555912+0000", 2024-09-12T07:19:57.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6150000000000001e-06 2024-09-12T07:19:57.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555923+0000", 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0832000000000001e-05 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.555930+0000", 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7279999999999995e-06 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556075+0000", 2024-09-12T07:19:57.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014423300000000001 2024-09-12T07:19:57.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556146+0000", 2024-09-12T07:19:57.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1710000000000003e-05 2024-09-12T07:19:57.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:57.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556235+0000", 2024-09-12T07:19:57.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9161999999999998e-05 2024-09-12T07:19:57.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556559+0000", 2024-09-12T07:19:57.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000323407 2024-09-12T07:19:57.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558730+0000", 2024-09-12T07:19:57.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021707190000000002 2024-09-12T07:19:57.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558761+0000", 2024-09-12T07:19:57.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1504000000000002e-05 2024-09-12T07:19:57.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558840+0000", 2024-09-12T07:19:57.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8986000000000006e-05 2024-09-12T07:19:57.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.330 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:493 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.556112+0000", 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.7586985049999999, 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029569990000000001, 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 493 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556112+0000", 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556112+0000", 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556113+0000", 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6799999999999999e-07 2024-09-12T07:19:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556117+0000", 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2239999999999997e-06 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556125+0000", 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9799999999999998e-06 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556129+0000", 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1559999999999997e-06 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556391+0000", 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026192499999999998 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for rw locks", 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.556429+0000", 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7280000000000002e-05 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558887+0000", 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024582580000000001 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.558980+0000", 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3177999999999995e-05 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.559069+0000", 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9229999999999998e-05 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.336 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:610 3.9 3:911399f5:::rbd_data.116011b0bca6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:57.336 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.685417+0000", 2024-09-12T07:19:57.336 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.62939349, 2024-09-12T07:19:57.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001501474, 2024-09-12T07:19:57.336 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.337 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.337 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.337 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 610 2024-09-12T07:19:57.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.337 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685417+0000", 2024-09-12T07:19:57.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685417+0000", 2024-09-12T07:19:57.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685419+0000", 2024-09-12T07:19:57.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.739e-06 2024-09-12T07:19:57.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685427+0000", 2024-09-12T07:19:57.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8890000000000002e-06 2024-09-12T07:19:57.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685443+0000", 2024-09-12T07:19:57.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6586999999999999e-05 2024-09-12T07:19:57.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685450+0000", 2024-09-12T07:19:57.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0729999999999999e-06 2024-09-12T07:19:57.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685511+0000", 2024-09-12T07:19:57.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0596000000000001e-05 2024-09-12T07:19:57.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685613+0000", 2024-09-12T07:19:57.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010168 2024-09-12T07:19:57.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:57.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.685756+0000", 2024-09-12T07:19:57.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000143237 2024-09-12T07:19:57.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686751+0000", 2024-09-12T07:19:57.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099451300000000004 2024-09-12T07:19:57.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686802+0000", 2024-09-12T07:19:57.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0998000000000002e-05 2024-09-12T07:19:57.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686873+0000", 2024-09-12T07:19:57.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1029e-05 2024-09-12T07:19:57.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686919+0000", 2024-09-12T07:19:57.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6133e-05 2024-09-12T07:19:57.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.349 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.350 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:956 3.9 3:96f4f606:::rbd_data.116011b0bca6.0000000000000027:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:57.350 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.206153+0000", 2024-09-12T07:19:57.350 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1086574500000008, 2024-09-12T07:19:57.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016258261999999999, 2024-09-12T07:19:57.351 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.351 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.351 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.351 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.351 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 956 2024-09-12T07:19:57.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.352 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206153+0000", 2024-09-12T07:19:57.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206153+0000", 2024-09-12T07:19:57.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206155+0000", 2024-09-12T07:19:57.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0499999999999999e-06 2024-09-12T07:19:57.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206163+0000", 2024-09-12T07:19:57.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6699999999999994e-06 2024-09-12T07:19:57.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206181+0000", 2024-09-12T07:19:57.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7645000000000001e-05 2024-09-12T07:19:57.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206188+0000", 2024-09-12T07:19:57.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0829999999999998e-06 2024-09-12T07:19:57.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206239+0000", 2024-09-12T07:19:57.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0973000000000001e-05 2024-09-12T07:19:57.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.206346+0000", 2024-09-12T07:19:57.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010729400000000001 2024-09-12T07:19:57.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:57.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.209070+0000", 2024-09-12T07:19:57.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027238340000000001 2024-09-12T07:19:57.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222329+0000", 2024-09-12T07:19:57.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013259655 2024-09-12T07:19:57.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222341+0000", 2024-09-12T07:19:57.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.119e-05 2024-09-12T07:19:57.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222378+0000", 2024-09-12T07:19:57.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7008000000000002e-05 2024-09-12T07:19:57.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222411+0000", 2024-09-12T07:19:57.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3859999999999998e-05 2024-09-12T07:19:57.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.360 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.360 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:956 3.9 e28/24 ondisk, result = 0)", 2024-09-12T07:19:57.360 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.217233+0000", 2024-09-12T07:19:57.361 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.097577802, 2024-09-12T07:19:57.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051019610000000003, 2024-09-12T07:19:57.361 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.362 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217233+0000", 2024-09-12T07:19:57.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217233+0000", 2024-09-12T07:19:57.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217234+0000", 2024-09-12T07:19:57.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.579e-06 2024-09-12T07:19:57.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217243+0000", 2024-09-12T07:19:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3310000000000004e-06 2024-09-12T07:19:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217258+0000", 2024-09-12T07:19:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4715000000000001e-05 2024-09-12T07:19:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.217267+0000", 2024-09-12T07:19:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1439999999999992e-06 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222316+0000", 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050498039999999998 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222325+0000", 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5480000000000002e-06 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.222335+0000", 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8400000000000007e-06 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:960 3.9 3:911399f5:::rbd_data.116011b0bca6.0000000000000007:head [stat,write 228792~62787] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.236487+0000", 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.0783237759999995, 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016742370000000001, 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 960 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236487+0000", 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236487+0000", 2024-09-12T07:19:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236489+0000", 2024-09-12T07:19:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.686e-06 2024-09-12T07:19:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236510+0000", 2024-09-12T07:19:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1060000000000002e-05 2024-09-12T07:19:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236522+0000", 2024-09-12T07:19:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.256e-05 2024-09-12T07:19:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236528+0000", 2024-09-12T07:19:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9150000000000001e-06 2024-09-12T07:19:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236588+0000", 2024-09-12T07:19:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9398999999999998e-05 2024-09-12T07:19:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236692+0000", 2024-09-12T07:19:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104589 2024-09-12T07:19:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.236857+0000", 2024-09-12T07:19:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016502699999999999 2024-09-12T07:19:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.237713+0000", 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00085576399999999996 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.238083+0000", 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037034999999999998 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.238115+0000", 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2211e-05 2024-09-12T07:19:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.238161+0000", 2024-09-12T07:19:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5676000000000001e-05 2024-09-12T07:19:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:984 3.9 3:96f4f606:::rbd_data.116011b0bca6.0000000000000027:head [write 1013176~35400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.269836+0000", 2024-09-12T07:19:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.0449745509999993, 2024-09-12T07:19:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020110530000000001, 2024-09-12T07:19:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 984 2024-09-12T07:19:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269836+0000", 2024-09-12T07:19:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269836+0000", 2024-09-12T07:19:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269838+0000", 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7770000000000001e-06 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269857+0000", 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9029999999999999e-05 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269874+0000", 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7359000000000001e-05 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269882+0000", 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.269938+0000", 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5801999999999999e-05 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.270002+0000", 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4448000000000004e-05 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.270086+0000", 2024-09-12T07:19:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3919999999999996e-05 2024-09-12T07:19:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.270707+0000", 2024-09-12T07:19:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00062067299999999997 2024-09-12T07:19:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.271735+0000", 2024-09-12T07:19:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010287720000000001 2024-09-12T07:19:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.271796+0000", 2024-09-12T07:19:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0791000000000001e-05 2024-09-12T07:19:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.271847+0000", 2024-09-12T07:19:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1070999999999999e-05 2024-09-12T07:19:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1111 3.9 3:97eec03a:::rbd_header.1160e8dffb2f:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.493764+0000", 2024-09-12T07:19:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8210464809999998, 2024-09-12T07:19:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020530140000000001, 2024-09-12T07:19:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1111 2024-09-12T07:19:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493764+0000", 2024-09-12T07:19:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493764+0000", 2024-09-12T07:19:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493766+0000", 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6440000000000001e-06 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493774+0000", 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0749999999999998e-06 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493791+0000", 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6799e-05 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493801+0000", 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9939999999999995e-06 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493859+0000", 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8269000000000001e-05 2024-09-12T07:19:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.493864+0000", 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4970000000000001e-06 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.495607+0000", 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001742586 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.495670+0000", 2024-09-12T07:19:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3077000000000006e-05 2024-09-12T07:19:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.495817+0000", 2024-09-12T07:19:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014707300000000001 2024-09-12T07:19:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1345 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7340032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.755762+0000", 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.5590486569999999, 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0066881680000000004, 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1345 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755762+0000", 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755762+0000", 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755764+0000", 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2199999999999999e-06 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756112+0000", 2024-09-12T07:19:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000347287 2024-09-12T07:19:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756136+0000", 2024-09-12T07:19:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4488e-05 2024-09-12T07:19:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756147+0000", 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0857e-05 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756221+0000", 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4200000000000001e-05 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756351+0000", 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012945 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756621+0000", 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027040400000000002 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759352+0000", 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027308290000000002 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.762330+0000", 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029781170000000002 2024-09-12T07:19:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.762400+0000", 2024-09-12T07:19:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0220000000000002e-05 2024-09-12T07:19:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.762450+0000", 2024-09-12T07:19:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0096000000000003e-05 2024-09-12T07:19:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1561 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:head [stat,truncate 1280616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.997537+0000", 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.3172736399999998, 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002027367, 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1561 2024-09-12T07:19:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997537+0000", 2024-09-12T07:19:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997537+0000", 2024-09-12T07:19:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997539+0000", 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.215e-06 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997550+0000", 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0763e-05 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997572+0000", 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1647000000000001e-05 2024-09-12T07:19:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997581+0000", 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8020000000000003e-06 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997650+0000", 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8845999999999998e-05 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997735+0000", 2024-09-12T07:19:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4915999999999998e-05 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.997872+0000", 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000136365 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.998859+0000", 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00098759400000000006 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999454+0000", 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00059463300000000001 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999523+0000", 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9289999999999996e-05 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.999564+0000", 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1295999999999999e-05 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1701 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:head [stat,truncate 524288] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.564930+0000", 2024-09-12T07:19:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.7498806509999998, 2024-09-12T07:19:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045744499999999999, 2024-09-12T07:19:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1701 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.564930+0000", 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.564930+0000", 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.564932+0000", 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9970000000000001e-06 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.564962+0000", 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0323000000000001e-05 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.564985+0000", 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2762999999999999e-05 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.564995+0000", 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8810000000000008e-06 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.565066+0000", 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1223999999999994e-05 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.565169+0000", 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010238699999999999 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.565386+0000", 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021692099999999999 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.566812+0000", 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001426229 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.569440+0000", 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026286249999999999 2024-09-12T07:19:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.569475+0000", 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4482e-05 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.569504+0000", 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9618e-05 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1923 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:5 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.803850+0000", 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5109605689999999, 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016816719999999999, 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1923 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803850+0000", 2024-09-12T07:19:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803850+0000", 2024-09-12T07:19:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803851+0000", 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.01e-07 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803852+0000", 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9319999999999999e-06 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803859+0000", 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803863+0000", 2024-09-12T07:19:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3669999999999999e-06 2024-09-12T07:19:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805353+0000", 2024-09-12T07:19:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014896760000000001 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805428+0000", 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4868000000000002e-05 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805532+0000", 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104248 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1924 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:5 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.803908+0000", 2024-09-12T07:19:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5109029869999997, 2024-09-12T07:19:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0018887890000000001, 2024-09-12T07:19:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1924 2024-09-12T07:19:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803908+0000", 2024-09-12T07:19:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803908+0000", 2024-09-12T07:19:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803908+0000", 2024-09-12T07:19:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9999999999999996e-07 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803913+0000", 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3760000000000001e-06 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803924+0000", 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1092e-05 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.803932+0000", 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6199999999999999e-06 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805574+0000", 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016424040000000001 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805673+0000", 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8936000000000003e-05 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.805796+0000", 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012356100000000001 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1959 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:5 [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.811258+0000", 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5035525639999996, 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016934879999999999, 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1959 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811258+0000", 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811258+0000", 2024-09-12T07:19:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811259+0000", 2024-09-12T07:19:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8100000000000003e-07 2024-09-12T07:19:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811261+0000", 2024-09-12T07:19:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8480000000000001e-06 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811268+0000", 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.024e-06 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811274+0000", 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3170000000000001e-06 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.812750+0000", 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014765710000000001 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.812836+0000", 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5484999999999999e-05 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.812952+0000", 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011576199999999999 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1960 3.9 3:940b4aaa:::rbd_data.116075042ea1.0000000000000007:5 [sparse-read 7340032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.811282+0000", 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5035291009999998, 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019447640000000001, 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1960 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811282+0000", 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811282+0000", 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811282+0000", 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4700000000000002e-07 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811292+0000", 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8490000000000001e-06 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811299+0000", 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8399999999999997e-06 2024-09-12T07:19:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.811304+0000", 2024-09-12T07:19:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1499999999999998e-06 2024-09-12T07:19:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.813019+0000", 2024-09-12T07:19:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017152549999999999 2024-09-12T07:19:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.813112+0000", 2024-09-12T07:19:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2807999999999996e-05 2024-09-12T07:19:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.813226+0000", 2024-09-12T07:19:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011441499999999999 2024-09-12T07:19:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1995 3.9 3:951701e1:::rbd_data.1160947fa40f.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.844656+0000", 2024-09-12T07:19:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.4701550980000002, 2024-09-12T07:19:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028069369999999998, 2024-09-12T07:19:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1995 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844656+0000", 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844656+0000", 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844657+0000", 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.722e-06 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844664+0000", 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8059999999999997e-06 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844680+0000", 2024-09-12T07:19:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6181999999999999e-05 2024-09-12T07:19:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844688+0000", 2024-09-12T07:19:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2690000000000002e-06 2024-09-12T07:19:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844752+0000", 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4471e-05 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844847+0000", 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4857000000000003e-05 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.844971+0000", 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012368000000000001 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.845521+0000", 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00055040299999999998 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.847359+0000", 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0018378330000000001 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.847416+0000", 2024-09-12T07:19:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7661999999999998e-05 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.847463+0000", 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6051999999999999e-05 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2907 3.9 3:96683ca3:::rbd_data.11603e8faf82.0000000000000059:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:53.557277+0000", 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.7575336940000001, 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065074269999999997, 2024-09-12T07:19:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2907 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.557277+0000", 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.557277+0000", 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.557279+0000", 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9980000000000002e-06 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.557285+0000", 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6640000000000003e-06 2024-09-12T07:19:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.557301+0000", 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6235999999999998e-05 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.557309+0000", 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0590000000000003e-06 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.557418+0000", 2024-09-12T07:19:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109539 2024-09-12T07:19:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.557550+0000", 2024-09-12T07:19:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013105300000000001 2024-09-12T07:19:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.563784+0000", 2024-09-12T07:19:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062348780000000001 2024-09-12T07:19:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:6412 3.9 3:953c2176:::rbd_data.11603e8faf82.000000000000005b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:53.807216+0000", 2024-09-12T07:19:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5075945549999998, 2024-09-12T07:19:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001723294, 2024-09-12T07:19:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6412 2024-09-12T07:19:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.807216+0000", 2024-09-12T07:19:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.807216+0000", 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.807218+0000", 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.468e-06 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.807225+0000", 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9600000000000003e-06 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.807238+0000", 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3675999999999999e-05 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.807247+0000", 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3659999999999992e-06 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.807311+0000", 2024-09-12T07:19:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4039000000000001e-05 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.807394+0000", 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3826999999999995e-05 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.807643+0000", 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024870099999999999 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.808273+0000", 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00063022800000000004 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.808827+0000", 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00055345500000000001 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.808894+0000", 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7253000000000001e-05 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.808939+0000", 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5321000000000002e-05 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:6506 3.9 3:96683ca3:::rbd_data.11603e8faf82.0000000000000059:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:53.825798+0000", 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.4890128489999999, 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015239780000000001, 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6506 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.825798+0000", 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.825798+0000", 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.825799+0000", 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.825804+0000", 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1950000000000002e-06 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.825816+0000", 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2165e-05 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.825822+0000", 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9170000000000003e-06 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.825881+0000", 2024-09-12T07:19:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8938999999999999e-05 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.825989+0000", 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107268 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.826231+0000", 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000242031 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.826928+0000", 2024-09-12T07:19:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00069729999999999998 2024-09-12T07:19:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.827265+0000", 2024-09-12T07:19:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000336707 2024-09-12T07:19:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.827298+0000", 2024-09-12T07:19:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3234000000000001e-05 2024-09-12T07:19:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:53.827322+0000", 2024-09-12T07:19:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3980000000000001e-05 2024-09-12T07:19:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.450 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:57.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:19:57.454 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:19:57.454 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:19:57.454 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:19:57.455 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:19:57.455 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:19:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:19:57.456 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:57.454+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:57.552 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:19:57.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-12T07:19:57.553 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:57.554+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:57.603 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:19:57.603 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:57.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:19:57.653 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:57.654+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:57.654 INFO:teuthology.orchestra.run.smithi046.stdout:732 write 0xae9df84 thru 0xaeac81e (0xe89b bytes) 2024-09-12T07:19:57.659 INFO:teuthology.orchestra.run.smithi046.stdout:734 writesame 0x4091e1b thru 0x409fd1f data_size 0x1(0xdf05 bytes) 2024-09-12T07:19:57.688 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:57.688 INFO:teuthology.orchestra.run.smithi046.stdout:736 read 0x1e0b4b8 thru 0x1e11541 (0x608a bytes) 2024-09-12T07:19:57.697 INFO:teuthology.orchestra.run.smithi046.stdout:737 writesame 0xd0f39e8 thru 0xd0fdff6 data_size 0x1(0xa60f bytes) 2024-09-12T07:19:57.698 INFO:teuthology.orchestra.run.smithi046.stdout:738 read 0x7ba36d3 thru 0x7baed75 (0xb6a3 bytes) 2024-09-12T07:19:57.698 INFO:teuthology.orchestra.run.smithi046.stdout:741 punch from 0x3f037a5 to 0x3f044ed, (0xd48 bytes) 2024-09-12T07:19:57.698 INFO:teuthology.orchestra.run.smithi046.stdout:743 punch from 0x23cb904 to 0x23d96b8, (0xddb4 bytes) 2024-09-12T07:19:57.698 INFO:teuthology.orchestra.run.smithi046.stdout:744 writesame 0x2a1dbcc thru 0x2a20ad4 data_size 0x1(0x2f09 bytes) 2024-09-12T07:19:57.702 INFO:teuthology.orchestra.run.smithi046.stdout:747 write 0xd01e825 thru 0xd025775 (0x6f51 bytes) 2024-09-12T07:19:57.706 INFO:teuthology.orchestra.run.smithi046.stdout:749 punch from 0x66bfaa5 to 0x66c0ad4, (0x102f bytes) 2024-09-12T07:19:57.706 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:57.706 INFO:teuthology.orchestra.run.smithi046.stdout:751 write 0x27fb87 thru 0x2822bb (0x2735 bytes) 2024-09-12T07:19:57.710 INFO:teuthology.orchestra.run.smithi046.stdout:752 read 0x648a556 thru 0x649872a (0xe1d5 bytes) 2024-09-12T07:19:57.710 INFO:teuthology.orchestra.run.smithi046.stdout:753 punch from 0x17ec1de to 0x17f3f01, (0x7d23 bytes) 2024-09-12T07:19:57.710 INFO:teuthology.orchestra.run.smithi046.stdout:754 punch from 0x945e604 to 0x9466e21, (0x881d bytes) 2024-09-12T07:19:57.710 INFO:teuthology.orchestra.run.smithi046.stdout:755 read 0x254437c thru 0x254fc08 (0xb88d bytes) 2024-09-12T07:19:57.711 INFO:teuthology.orchestra.run.smithi046.stdout:756 writesame 0x1da22a5 thru 0x1da79d3 data_size 0x1(0x572f bytes) 2024-09-12T07:19:57.722 INFO:teuthology.orchestra.run.smithi046.stdout:757 write 0xc2b87cd thru 0xc2b96bc (0xef0 bytes) 2024-09-12T07:19:57.724 INFO:teuthology.orchestra.run.smithi046.stdout:758 read 0xb4a9b18 thru 0xb4b4e27 (0xb310 bytes) 2024-09-12T07:19:57.724 INFO:teuthology.orchestra.run.smithi046.stdout:761 punch from 0x2f7067a to 0x2f7f18d, (0xeb13 bytes) 2024-09-12T07:19:57.730 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:57.730 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:19:57.731 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:19:57.731 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:19:57.731 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:57.736 INFO:teuthology.orchestra.run.smithi046.stdout:762 trunc from 0xd0fdff7 to 0x303eaa0 2024-09-12T07:19:57.743 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:19:57.755 INFO:teuthology.orchestra.run.smithi046.stdout:763 punch from 0x175b629 to 0x1760349, (0x4d20 bytes) 2024-09-12T07:19:57.755 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:57.756+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:57.757 INFO:teuthology.orchestra.run.smithi046.stdout:766 trunc from 0x303eaa0 to 0x1c06334 2024-09-12T07:19:57.766 INFO:teuthology.orchestra.run.smithi046.stdout:767 trunc from 0x1c06334 to 0x1badf78 2024-09-12T07:19:57.778 INFO:teuthology.orchestra.run.smithi046.stdout:768 read 0xc14cdd thru 0xc1d2fb (0x861f bytes) 2024-09-12T07:19:57.800 INFO:teuthology.orchestra.run.smithi046.stdout:769 write 0xd19fab1 thru 0xd1a7a6d (0x7fbd bytes) 2024-09-12T07:19:57.803 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:57.803 INFO:teuthology.orchestra.run.smithi046.stdout:774 write 0xae8e29c thru 0xae8e2a9 (0xe bytes) 2024-09-12T07:19:57.813 INFO:teuthology.orchestra.run.smithi046.stdout:776 write 0xeb1c930 thru 0xeb23531 (0x6c02 bytes) 2024-09-12T07:19:57.815 INFO:teuthology.orchestra.run.smithi046.stdout:777 punch from 0x5d05aa7 to 0x5d150f2, (0xf64b bytes) 2024-09-12T07:19:57.815 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:57.815 INFO:teuthology.orchestra.run.smithi046.stdout:781 writesame 0x5944509 thru 0x59512b6 data_size 0x7(0xcdae bytes) 2024-09-12T07:19:57.821 INFO:teuthology.orchestra.run.smithi046.stdout:784 write 0xe88ecf8 thru 0xe89b206 (0xc50f bytes) 2024-09-12T07:19:57.824 INFO:teuthology.orchestra.run.smithi046.stdout:785 read 0x3ea84ef thru 0x3eb54f3 (0xd005 bytes) 2024-09-12T07:19:57.825 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:57.828 INFO:teuthology.orchestra.run.smithi046.stdout:787 trunc from 0xeb23532 to 0xc32a7aa 2024-09-12T07:19:57.835 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:57.835 INFO:teuthology.orchestra.run.smithi046.stdout:789 write 0x46b4d53 thru 0x46b77d9 (0x2a87 bytes) 2024-09-12T07:19:57.838 INFO:teuthology.orchestra.run.smithi046.stdout:792 read 0x101322e thru 0x101586c (0x263f bytes) 2024-09-12T07:19:57.839 INFO:teuthology.orchestra.run.smithi046.stdout:793 read 0x1e5d3c4 thru 0x1e6d177 (0xfdb4 bytes) 2024-09-12T07:19:57.839 INFO:teuthology.orchestra.run.smithi046.stdout:794 writesame 0x75edb1c thru 0x75ee8da data_size 0x17(0xdbf bytes) 2024-09-12T07:19:57.842 INFO:teuthology.orchestra.run.smithi046.stdout:795 punch from 0x3885463 to 0x38870e3, (0x1c80 bytes) 2024-09-12T07:19:57.851 INFO:teuthology.orchestra.run.smithi046.stdout:796 trunc from 0xc32a7aa to 0x42b3208 2024-09-12T07:19:57.854 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:57.855+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:57.866 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:57.866 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:57.866 INFO:teuthology.orchestra.run.smithi046.stdout:truncating to largest ever: 0xed2a02e 2024-09-12T07:19:57.879 INFO:teuthology.orchestra.run.smithi046.stdout:803 trunc from 0x42b3208 to 0xed2a02e 2024-09-12T07:19:57.882 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:19:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:19:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:19:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:19:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:613 3.7 3:e175feac:::rbd_data.116011b0bca6.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.686095+0000", 2024-09-12T07:19:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.196376817999999, 2024-09-12T07:19:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054618599999999998, 2024-09-12T07:19:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 613 2024-09-12T07:19:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686095+0000", 2024-09-12T07:19:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686095+0000", 2024-09-12T07:19:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686097+0000", 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0699999999999999e-06 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686102+0000", 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6570000000000002e-06 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686114+0000", 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1637999999999999e-05 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686123+0000", 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6789999999999994e-06 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686187+0000", 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4734999999999996e-05 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686337+0000", 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000149383 2024-09-12T07:19:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.686544+0000", 2024-09-12T07:19:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020754000000000001 2024-09-12T07:19:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687923+0000", 2024-09-12T07:19:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013792850000000001 2024-09-12T07:19:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691480+0000", 2024-09-12T07:19:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035560660000000001 2024-09-12T07:19:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691514+0000", 2024-09-12T07:19:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3979000000000001e-05 2024-09-12T07:19:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691557+0000", 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3828000000000001e-05 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:617 3.2 3:4522120e:::rbd_data.116011b0bca6.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:47.687024+0000", 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.195447981999999, 2024-09-12T07:19:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047272190000000004, 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 617 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687024+0000", 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687024+0000", 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687025+0000", 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6000000000000002e-07 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687055+0000", 2024-09-12T07:19:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.029e-05 2024-09-12T07:19:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687065+0000", 2024-09-12T07:19:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6749999999999997e-06 2024-09-12T07:19:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687072+0000", 2024-09-12T07:19:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.657e-06 2024-09-12T07:19:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687131+0000", 2024-09-12T07:19:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9157000000000003e-05 2024-09-12T07:19:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687263+0000", 2024-09-12T07:19:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013171799999999999 2024-09-12T07:19:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.687458+0000", 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019529899999999999 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.688377+0000", 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00091874700000000003 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691642+0000", 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0032655290000000001 2024-09-12T07:19:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691706+0000", 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3548999999999998e-05 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:47.691752+0000", 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5738000000000002e-05 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:894 3.2 3:468e8e09:::rbd_data.116011b0bca6.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:19:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.7795603740000008, 2024-09-12T07:19:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015762987999999999, 2024-09-12T07:19:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 894 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102913+0000", 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004e-06 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102918+0000", 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6260000000000003e-06 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102929+0000", 2024-09-12T07:19:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1382e-05 2024-09-12T07:19:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102934+0000", 2024-09-12T07:19:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:19:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102988+0000", 2024-09-12T07:19:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3683000000000001e-05 2024-09-12T07:19:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.103072+0000", 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4135000000000007e-05 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.105687+0000", 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026149580000000001 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118570+0000", 2024-09-12T07:19:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012883034 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118585+0000", 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4797000000000001e-05 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118622+0000", 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7163000000000003e-05 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118675+0000", 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3236000000000003e-05 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:894 3.2 e28/24 ondisk, result = 0)", 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.112622+0000", 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.7698507960000001, 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059558010000000002, 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112622+0000", 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112622+0000", 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112623+0000", 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9800000000000002e-07 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112627+0000", 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0720000000000001e-06 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112640+0000", 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3131e-05 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.112649+0000", 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8710000000000003e-06 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118557+0000", 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059084740000000004 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118565+0000", 2024-09-12T07:19:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5270000000000001e-06 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118577+0000", 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2728000000000001e-05 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:912 3.2 3:468e8e09:::rbd_data.116011b0bca6.000000000000003e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.143632+0000", 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.7388400930000003, 2024-09-12T07:19:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051497230000000001, 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 912 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143632+0000", 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143632+0000", 2024-09-12T07:19:57.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143634+0000", 2024-09-12T07:19:57.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0040000000000002e-06 2024-09-12T07:19:57.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143641+0000", 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1199999999999996e-06 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143660+0000", 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8408999999999999e-05 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143670+0000", 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0312999999999999e-05 2024-09-12T07:19:57.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143747+0000", 2024-09-12T07:19:57.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7399999999999998e-05 2024-09-12T07:19:57.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.143855+0000", 2024-09-12T07:19:57.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010739999999999999 2024-09-12T07:19:57.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:19:57.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.144004+0000", 2024-09-12T07:19:57.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014922500000000001 2024-09-12T07:19:57.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.144699+0000", 2024-09-12T07:19:57.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00069510800000000005 2024-09-12T07:19:57.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.148690+0000", 2024-09-12T07:19:57.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039904490000000001 2024-09-12T07:19:57.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.148741+0000", 2024-09-12T07:19:57.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1328000000000001e-05 2024-09-12T07:19:57.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.148782+0000", 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:993 3.7 3:e61faebc:::rbd_data.116011b0bca6.000000000000004c:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.295319+0000", 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5871534860000001, 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079320569999999993, 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.914 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.914 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 993 2024-09-12T07:19:57.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.914 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295319+0000", 2024-09-12T07:19:57.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295319+0000", 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295321+0000", 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7e-06 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295328+0000", 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2729999999999997e-06 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295344+0000", 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6033e-05 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295353+0000", 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2699999999999993e-06 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295409+0000", 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5507000000000002e-05 2024-09-12T07:19:57.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.295521+0000", 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112719 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.296644+0000", 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001123045 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303082+0000", 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064377510000000002 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303131+0000", 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9295999999999997e-05 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303195+0000", 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4022999999999993e-05 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.303251+0000", 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5439999999999998e-05 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1031 3.7 3:e61faebc:::rbd_data.116011b0bca6.000000000000004c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.377551+0000", 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5049212920000006, 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0050361379999999999, 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.919 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1031 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377551+0000", 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377551+0000", 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377554+0000", 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7970000000000001e-06 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377565+0000", 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1612e-05 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377589+0000", 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3955999999999999e-05 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377603+0000", 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3540000000000001e-05 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377698+0000", 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5172000000000003e-05 2024-09-12T07:19:57.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377793+0000", 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5137999999999996e-05 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.377920+0000", 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126518 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.378626+0000", 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00070584200000000004 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.382513+0000", 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038871539999999999 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.382542+0000", 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9583000000000001e-05 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.382587+0000", 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4826000000000001e-05 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1080 3.b 3:d0645053:::rbd_object_map.116011b0bca6.0000000000000004:head [writefull 0~53] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.470376+0000", 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.4120962939999995, 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029172759999999999, 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.925 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1080 2024-09-12T07:19:57.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.926 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470376+0000", 2024-09-12T07:19:57.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470376+0000", 2024-09-12T07:19:57.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470379+0000", 2024-09-12T07:19:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.571e-06 2024-09-12T07:19:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470388+0000", 2024-09-12T07:19:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1279999999999997e-06 2024-09-12T07:19:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470405+0000", 2024-09-12T07:19:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7229000000000001e-05 2024-09-12T07:19:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470416+0000", 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0781000000000001e-05 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470489+0000", 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3366000000000002e-05 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.470497+0000", 2024-09-12T07:19:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0180000000000001e-06 2024-09-12T07:19:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.472041+0000", 2024-09-12T07:19:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015442780000000001 2024-09-12T07:19:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.472118+0000", 2024-09-12T07:19:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6599000000000003e-05 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.472198+0000", 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9742999999999999e-05 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.472726+0000", 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052847500000000004 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.473232+0000", 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00050537100000000001 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.473262+0000", 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0827999999999997e-05 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.473293+0000", 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0889000000000002e-05 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1143 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.514700+0000", 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.3677718409999997, 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029554020000000002, 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1143 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514700+0000", 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514700+0000", 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514701+0000", 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2900000000000003e-07 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514705+0000", 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8260000000000003e-06 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514713+0000", 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3440000000000001e-06 2024-09-12T07:19:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514717+0000", 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9500000000000003e-06 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514757+0000", 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9672999999999998e-05 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514799+0000", 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1978999999999999e-05 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.514927+0000", 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012807000000000001 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.515330+0000", 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040297199999999998 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.517605+0000", 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022750610000000001 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.517634+0000", 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.853e-05 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.517656+0000", 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2268000000000001e-05 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1338 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.748194+0000", 2024-09-12T07:19:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1342780440000002, 2024-09-12T07:19:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038514669999999999, 2024-09-12T07:19:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1338 2024-09-12T07:19:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748194+0000", 2024-09-12T07:19:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748194+0000", 2024-09-12T07:19:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748196+0000", 2024-09-12T07:19:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4759999999999999e-06 2024-09-12T07:19:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748647+0000", 2024-09-12T07:19:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00045091599999999999 2024-09-12T07:19:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748661+0000", 2024-09-12T07:19:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4534e-05 2024-09-12T07:19:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748668+0000", 2024-09-12T07:19:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9009999999999997e-06 2024-09-12T07:19:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748723+0000", 2024-09-12T07:19:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4468000000000001e-05 2024-09-12T07:19:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748830+0000", 2024-09-12T07:19:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010779 2024-09-12T07:19:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.748996+0000", 2024-09-12T07:19:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165162 2024-09-12T07:19:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.751387+0000", 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002391706 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.751907+0000", 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051961499999999996 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.751981+0000", 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4319000000000005e-05 2024-09-12T07:19:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.752046+0000", 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4579999999999995e-05 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1343 3.7 3:e78f2392:::rbd_data.116075042ea1.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7340032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.755402+0000", 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1270700569999992, 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0040308560000000002, 2024-09-12T07:19:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1343 2024-09-12T07:19:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755402+0000", 2024-09-12T07:19:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755402+0000", 2024-09-12T07:19:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755405+0000", 2024-09-12T07:19:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.937e-06 2024-09-12T07:19:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755456+0000", 2024-09-12T07:19:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0642000000000001e-05 2024-09-12T07:19:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755479+0000", 2024-09-12T07:19:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3527e-05 2024-09-12T07:19:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755492+0000", 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3132999999999999e-05 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755569+0000", 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6323999999999996e-05 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755722+0000", 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000153394 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755924+0000", 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020157500000000001 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.757983+0000", 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00205894 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759317+0000", 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013344229999999999 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759382+0000", 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4969999999999996e-05 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759433+0000", 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0991e-05 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1344 3.18 3:1a9d4b56:::rbd_data.116075042ea1.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7340032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.755621+0000", 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1268512800000003, 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0040014960000000002, 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1344 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755621+0000", 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755621+0000", 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755622+0000", 2024-09-12T07:19:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2600000000000001e-07 2024-09-12T07:19:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755847+0000", 2024-09-12T07:19:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022498 2024-09-12T07:19:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755858+0000", 2024-09-12T07:19:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1426e-05 2024-09-12T07:19:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755866+0000", 2024-09-12T07:19:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6559999999999992e-06 2024-09-12T07:19:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755973+0000", 2024-09-12T07:19:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010666300000000001 2024-09-12T07:19:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756091+0000", 2024-09-12T07:19:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011859800000000001 2024-09-12T07:19:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:19:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756678+0000", 2024-09-12T07:19:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00058683100000000001 2024-09-12T07:19:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759291+0000", 2024-09-12T07:19:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002612582 2024-09-12T07:19:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759551+0000", 2024-09-12T07:19:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026036000000000001 2024-09-12T07:19:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759585+0000", 2024-09-12T07:19:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3516999999999998e-05 2024-09-12T07:19:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759623+0000", 2024-09-12T07:19:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7957000000000003e-05 2024-09-12T07:19:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:1341 3.19 e29/24 3:98c9a0e4:::rbd_data.116075042ea1.0000000000000004:head v 29'99, mlcod=28'97)", 2024-09-12T07:19:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.755920+0000", 2024-09-12T07:19:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1265519019999992, 2024-09-12T07:19:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031212290000000001, 2024-09-12T07:19:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755920+0000", 2024-09-12T07:19:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755920+0000", 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755921+0000", 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6699999999999999e-07 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755950+0000", 2024-09-12T07:19:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9479999999999999e-05 2024-09-12T07:19:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755961+0000", 2024-09-12T07:19:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0352e-05 2024-09-12T07:19:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.755968+0000", 2024-09-12T07:19:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0890000000000002e-06 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756047+0000", 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9684000000000005e-05 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.756069+0000", 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1795999999999999e-05 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.758791+0000", 2024-09-12T07:19:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002721472 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.759042+0000", 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00025088900000000002 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:1587 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.471457+0000", 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.4110151680000005, 2024-09-12T07:19:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031074520000000001, 2024-09-12T07:19:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1587 2024-09-12T07:19:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471457+0000", 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471457+0000", 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471459+0000", 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.728e-06 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471470+0000", 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1061e-05 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471487+0000", 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7333000000000002e-05 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471496+0000", 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7430000000000008e-06 2024-09-12T07:19:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471561+0000", 2024-09-12T07:19:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4633000000000003e-05 2024-09-12T07:19:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-12T07:19:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.471567+0000", 2024-09-12T07:19:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0630000000000002e-06 2024-09-12T07:19:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.473095+0000", 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001528092 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.473152+0000", 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7540999999999997e-05 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.473365+0000", 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021284599999999999 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.473824+0000", 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00045857500000000002 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.474486+0000", 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00066262099999999996 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.474520+0000", 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3392999999999997e-05 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.474565+0000", 2024-09-12T07:19:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4823000000000001e-05 2024-09-12T07:19:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2034 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:5 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.849522+0000", 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.0329505680000004, 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028828289999999999, 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2034 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849522+0000", 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849522+0000", 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849522+0000", 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0699999999999998e-07 2024-09-12T07:19:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849524+0000", 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0650000000000001e-06 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849530+0000", 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2609999999999999e-06 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849535+0000", 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7949999999999998e-06 2024-09-12T07:19:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852233+0000", 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026980350000000001 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852316+0000", 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2566999999999997e-05 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852405+0000", 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8698999999999999e-05 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2035 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:5 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.849543+0000", 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.0329297999999998, 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00306924, 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2035 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849543+0000", 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849543+0000", 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849543+0000", 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849545+0000", 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1270000000000001e-06 2024-09-12T07:19:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849552+0000", 2024-09-12T07:19:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6270000000000002e-06 2024-09-12T07:19:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849557+0000", 2024-09-12T07:19:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8749999999999999e-06 2024-09-12T07:19:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852435+0000", 2024-09-12T07:19:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028788580000000002 2024-09-12T07:19:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852515+0000", 2024-09-12T07:19:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9939999999999997e-05 2024-09-12T07:19:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852612+0000", 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6389000000000002e-05 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2036 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:5 [sparse-read 4718592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.849575+0000", 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.032897749, 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00318892, 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2036 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849575+0000", 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849575+0000", 2024-09-12T07:19:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849575+0000", 2024-09-12T07:19:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:19:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849578+0000", 2024-09-12T07:19:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8890000000000002e-06 2024-09-12T07:19:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849586+0000", 2024-09-12T07:19:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7149999999999998e-06 2024-09-12T07:19:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849591+0000", 2024-09-12T07:19:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6690000000000003e-06 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852640+0000", 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030482 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852689+0000", 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9181000000000002e-05 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852763+0000", 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4756000000000001e-05 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2037 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:5 [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.849600+0000", 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.0328727509999993, 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003393662, 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2037 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849600+0000", 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849600+0000", 2024-09-12T07:19:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849600+0000", 2024-09-12T07:19:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9499999999999998e-07 2024-09-12T07:19:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849602+0000", 2024-09-12T07:19:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:19:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849609+0000", 2024-09-12T07:19:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.533e-06 2024-09-12T07:19:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849617+0000", 2024-09-12T07:19:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6400000000000003e-06 2024-09-12T07:19:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852795+0000", 2024-09-12T07:19:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031781550000000002 2024-09-12T07:19:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852877+0000", 2024-09-12T07:19:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1249999999999996e-05 2024-09-12T07:19:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.852993+0000", 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116663 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2038 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:5 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.849626+0000", 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.032846267, 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003577233, 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2038 2024-09-12T07:19:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849626+0000", 2024-09-12T07:19:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849626+0000", 2024-09-12T07:19:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849626+0000", 2024-09-12T07:19:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3599999999999999e-07 2024-09-12T07:19:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849629+0000", 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3309999999999998e-06 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849635+0000", 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2790000000000004e-06 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849640+0000", 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230000000000001e-06 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.853024+0000", 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0033838850000000001 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.853106+0000", 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2068000000000001e-05 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.853203+0000", 2024-09-12T07:19:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7311000000000004e-05 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:2039 3.b 3:d380e784:::rbd_data.116075042ea1.0000000000000000:5 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:49.849648+0000", 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.0328243320000006, 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038036390000000002, 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:19:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 2039 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849648+0000", 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849648+0000", 2024-09-12T07:19:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849648+0000", 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.03e-07 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849653+0000", 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1420000000000003e-06 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:19:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849659+0000", 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9489999999999999e-06 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.849664+0000", 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0019999999999997e-06 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.853242+0000", 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035776839999999998 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.853336+0000", 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4136999999999997e-05 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:49.853452+0000", 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115322 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:19:58.004 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:19:58.006 INFO:teuthology.orchestra.run.smithi046.stdout:804 punch from 0x53223f5 to 0x5330400, (0xe00b bytes) 2024-09-12T07:19:58.006 INFO:teuthology.orchestra.run.smithi046.stdout:806 write 0x8aee930 thru 0x8afbe37 (0xd508 bytes) 2024-09-12T07:19:58.006 INFO:teuthology.orchestra.run.smithi046.stdout:807 read 0x83c3824 thru 0x83cb7b8 (0x7f95 bytes) 2024-09-12T07:19:58.007 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:19:58.009 INFO:teuthology.orchestra.run.smithi046.stdout:808 trunc from 0xed2a02e to 0xcaec6eb 2024-09-12T07:19:58.010 INFO:teuthology.orchestra.run.smithi046.stdout:809 write 0xac0f271 thru 0xac1bef5 (0xcc85 bytes) 2024-09-12T07:19:58.010 INFO:teuthology.orchestra.run.smithi046.stdout:810 write 0xd49781b thru 0xd49b4b6 (0x3c9c bytes) 2024-09-12T07:19:58.010 INFO:teuthology.orchestra.run.smithi046.stdout:811 write 0x4d91a2a thru 0x4d92db6 (0x138d bytes) 2024-09-12T07:19:58.010 INFO:teuthology.orchestra.run.smithi046.stdout:812 writesame 0xc851589 thru 0xc85e8b3 data_size 0x1(0xd32b bytes) 2024-09-12T07:19:58.010 INFO:teuthology.orchestra.run.smithi046.stdout:813 read 0x344f2ed thru 0x34567f6 (0x750a bytes) 2024-09-12T07:19:58.010 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:58.010 INFO:teuthology.orchestra.run.smithi046.stdout:816 punch from 0x920898e to 0x920e37f, (0x59f1 bytes) 2024-09-12T07:19:58.010 INFO:teuthology.orchestra.run.smithi046.stdout:818 write 0x1efd809 thru 0x1f05648 (0x7e40 bytes) 2024-09-12T07:19:58.010 INFO:teuthology.orchestra.run.smithi046.stdout:819 read 0x6c51691 thru 0x6c5fe17 (0xe787 bytes) 2024-09-12T07:19:58.010 INFO:teuthology.orchestra.run.smithi046.stdout:822 writesame 0xba63530 thru 0xba672f3 data_size 0x4(0x3dc4 bytes) 2024-09-12T07:19:58.011 INFO:teuthology.orchestra.run.smithi046.stdout:823 writesame 0x86f0a1e thru 0x86f70b2 data_size 0x1(0x6695 bytes) 2024-09-12T07:19:58.011 INFO:teuthology.orchestra.run.smithi046.stdout:825 read 0x2eee9eb thru 0x2ef5506 (0x6b1c bytes) 2024-09-12T07:19:58.011 INFO:teuthology.orchestra.run.smithi046.stdout:826 write 0x2027354 thru 0x2031c93 (0xa940 bytes) 2024-09-12T07:19:58.011 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:58.011 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:58.011 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:58.011 INFO:teuthology.orchestra.run.smithi046.stdout:831 read 0x11a5831 thru 0x11a8b83 (0x3353 bytes) 2024-09-12T07:19:58.011 INFO:teuthology.orchestra.run.smithi046.stdout:832 write 0x3897a1b thru 0x389efa9 (0x758f bytes) 2024-09-12T07:19:58.011 INFO:teuthology.orchestra.run.smithi046.stdout:833 trunc from 0xd49b4b7 to 0xca37980 2024-09-12T07:19:58.011 INFO:teuthology.orchestra.run.smithi046.stdout:834 writesame 0xba731a6 thru 0xba805db data_size 0x2(0xd436 bytes) 2024-09-12T07:19:58.011 INFO:teuthology.orchestra.run.smithi046.stdout:835 write 0x8bf96f1 thru 0x8bfa2ba (0xbca bytes) 2024-09-12T07:19:58.011 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:58.012 INFO:teuthology.orchestra.run.smithi046.stdout:837 write 0xe2c25ef thru 0xe2d1625 (0xf037 bytes) 2024-09-12T07:19:58.012 INFO:teuthology.orchestra.run.smithi046.stdout:838 read 0x3d5985a thru 0x3d65c90 (0xc437 bytes) 2024-09-12T07:19:58.012 INFO:teuthology.orchestra.run.smithi046.stdout:839 read 0x466e24c thru 0x4676c0d (0x89c2 bytes) 2024-09-12T07:19:58.012 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:58.012 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:58.012 INFO:teuthology.orchestra.run.smithi046.stdout:842 write 0xc94fcda thru 0xc958d3d (0x9064 bytes) 2024-09-12T07:19:58.012 INFO:teuthology.orchestra.run.smithi046.stdout:843 trunc from 0xe2d1626 to 0x6d47fd6 2024-09-12T07:19:58.012 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:58.007+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:58.022 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:58.022 INFO:teuthology.orchestra.run.smithi046.stdout:845 write 0x253917c thru 0x2544471 (0xb2f6 bytes) 2024-09-12T07:19:58.029 INFO:teuthology.orchestra.run.smithi046.stdout:847 writesame 0x7359a15 thru 0x7369603 data_size 0x1(0xfbef bytes) 2024-09-12T07:19:58.038 INFO:teuthology.orchestra.run.smithi046.stdout:848 trunc from 0x7369604 to 0xdfcd7ed 2024-09-12T07:19:58.042 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:58.055 INFO:teuthology.orchestra.run.smithi046.stdout:850 trunc from 0xdfcd7ed to 0x272a29e 2024-09-12T07:19:58.072 INFO:teuthology.orchestra.run.smithi046.stdout:853 read 0x270415d thru 0x270d354 (0x91f8 bytes) 2024-09-12T07:19:58.087 INFO:teuthology.orchestra.run.smithi046.stdout:854 write 0xb95bd65 thru 0xb961699 (0x5935 bytes) 2024-09-12T07:19:58.090 INFO:teuthology.orchestra.run.smithi046.stdout:855 read 0x2a52113 thru 0x2a5bc87 (0x9b75 bytes) 2024-09-12T07:19:58.093 INFO:teuthology.orchestra.run.smithi046.stdout:856 trunc from 0xb96169a to 0xd5dbffd 2024-09-12T07:19:58.099 INFO:teuthology.orchestra.run.smithi046.stdout:857 read 0xca61471 thru 0xca648ba (0x344a bytes) 2024-09-12T07:19:58.099 INFO:teuthology.orchestra.run.smithi046.stdout:859 punch from 0x3f6700e to 0x3f6dfc1, (0x6fb3 bytes) 2024-09-12T07:19:58.099 INFO:teuthology.orchestra.run.smithi046.stdout:860 read 0x83c9b2e thru 0x83cab10 (0xfe3 bytes) 2024-09-12T07:19:58.099 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:58.099 INFO:teuthology.orchestra.run.smithi046.stdout:864 read 0x8509dca thru 0x85193d2 (0xf609 bytes) 2024-09-12T07:19:58.099 INFO:teuthology.orchestra.run.smithi046.stdout:865 writesame 0xbeaf484 thru 0xbebc270 data_size 0x11(0xcded bytes) 2024-09-12T07:19:58.104 INFO:teuthology.orchestra.run.smithi046.stdout:866 punch from 0x75d5201 to 0x75e3526, (0xe325 bytes) 2024-09-12T07:19:58.104 INFO:teuthology.orchestra.run.smithi046.stdout:867 write 0x3314f55 thru 0x331ed20 (0x9dcc bytes) 2024-09-12T07:19:58.105 INFO:teuthology.orchestra.run.smithi046.stdout:868 write 0xb7765af thru 0xb77dd82 (0x77d4 bytes) 2024-09-12T07:19:58.106 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:58.107+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:58.107 INFO:teuthology.orchestra.run.smithi046.stdout:869 write 0xc3881ce thru 0xc3940ed (0xbf20 bytes) 2024-09-12T07:19:58.110 INFO:teuthology.orchestra.run.smithi046.stdout:870 write 0x6fa3c4a thru 0x6fb00a8 (0xc45f bytes) 2024-09-12T07:19:58.120 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.0 to 1 (10000) 2024-09-12T07:19:58.120 INFO:teuthology.orchestra.run.smithi046.stdout:872 trunc from 0xd5dbffd to 0x6c07444 2024-09-12T07:19:58.122 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:19:58.122 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:58.132 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:19:58.134 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:19:58.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-12T07:19:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:873 write 0x91649e thru 0x923b29 (0xd68c bytes) 2024-09-12T07:19:58.145 INFO:teuthology.orchestra.run.smithi046.stdout:874 read 0x32001bb thru 0x3202ec9 (0x2d0f bytes) 2024-09-12T07:19:58.152 INFO:teuthology.orchestra.run.smithi046.stdout:876 trunc from 0x6c07444 to 0x1ec6353 2024-09-12T07:19:58.164 INFO:teuthology.orchestra.run.smithi046.stdout:877 read 0x15678f7 thru 0x156df34 (0x663e bytes) 2024-09-12T07:19:58.165 INFO:teuthology.orchestra.run.smithi046.stdout:878 read 0x20149c thru 0x207cf0 (0x6855 bytes) 2024-09-12T07:19:58.166 INFO:teuthology.orchestra.run.smithi046.stdout:879 read 0x12c34ea thru 0x12cf80d (0xc324 bytes) 2024-09-12T07:19:58.167 INFO:teuthology.orchestra.run.smithi046.stdout:880 punch from 0x12791df to 0x1286307, (0xd128 bytes) 2024-09-12T07:19:58.182 INFO:teuthology.orchestra.run.smithi046.stdout:881 write 0xab3bdcf thru 0xab43913 (0x7b45 bytes) 2024-09-12T07:19:58.186 INFO:teuthology.orchestra.run.smithi046.stdout:882 punch from 0x4dfdc20 to 0x4dff149, (0x1529 bytes) 2024-09-12T07:19:58.195 INFO:teuthology.orchestra.run.smithi046.stdout:883 writesame 0xda8f667 thru 0xda911d0 data_size 0x1d(0x1b6a bytes) 2024-09-12T07:19:58.195 INFO:teuthology.orchestra.run.smithi046.stdout:886 punch from 0xcb6ab44 to 0xcb6e64f, (0x3b0b bytes) 2024-09-12T07:19:58.195 INFO:teuthology.orchestra.run.smithi046.stdout:888 write 0xab2042e thru 0xab2fdfa (0xf9cd bytes) 2024-09-12T07:19:58.196 INFO:teuthology.orchestra.run.smithi046.stdout:889 writesame 0x587d70 thru 0x594bf6 data_size 0x53(0xce87 bytes) 2024-09-12T07:19:58.199 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:58.199 INFO:teuthology.orchestra.run.smithi046.stdout:893 read 0x41bfd2c thru 0x41c8800 (0x8ad5 bytes) 2024-09-12T07:19:58.199 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:19:58.206 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:58.208+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:58.209 INFO:teuthology.orchestra.run.smithi046.stdout:895 trunc from 0xda911d1 to 0x5247d0d 2024-09-12T07:19:58.219 INFO:teuthology.orchestra.run.smithi046.stdout:897 write 0x2432838 thru 0x243d003 (0xa7cc bytes) 2024-09-12T07:19:58.235 INFO:teuthology.orchestra.run.smithi046.stdout:898 write 0xc0e9cc7 thru 0xc0f3d2b (0xa065 bytes) 2024-09-12T07:19:58.238 INFO:teuthology.orchestra.run.smithi046.stdout:901 write 0xb613b19 thru 0xb61e5c7 (0xaaaf bytes) 2024-09-12T07:19:58.243 INFO:teuthology.orchestra.run.smithi046.stdout:903 write 0x7930d96 thru 0x793554f (0x47ba bytes) 2024-09-12T07:19:58.246 INFO:teuthology.orchestra.run.smithi046.stdout:907 write 0x2eccef2 thru 0x2ed0935 (0x3a44 bytes) 2024-09-12T07:19:58.248 INFO:teuthology.orchestra.run.smithi046.stdout:908 write 0x1fb2b8f thru 0x1fb86a8 (0x5b1a bytes) 2024-09-12T07:19:58.249 INFO:teuthology.orchestra.run.smithi046.stdout:909 punch from 0x7d98b35 to 0x7d9d4d9, (0x49a4 bytes) 2024-09-12T07:19:58.250 INFO:teuthology.orchestra.run.smithi046.stdout:911 clone 6 order 23 su 2097152 sc 10 2024-09-12T07:19:58.283 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:19:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:19:58.319 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:58.321 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:19:58.321 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:19:58.322+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:19:58.421 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:19:58.423+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:19:58.429 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:19:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:19:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:19:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:592 3.f e28/24 3:f4a2090e:::rbd_object_map.116011b0bca6:head v 28'17, mlcod=28'14)", 2024-09-12T07:19:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.678290+0000", 2024-09-12T07:19:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 10.751857009, 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0067778999999999999, 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678290+0000", 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678290+0000", 2024-09-12T07:19:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678291+0000", 2024-09-12T07:19:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.68e-07 2024-09-12T07:19:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678313+0000", 2024-09-12T07:19:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1888000000000001e-05 2024-09-12T07:19:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678348+0000", 2024-09-12T07:19:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5063e-05 2024-09-12T07:19:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678373+0000", 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4973e-05 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678448+0000", 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5743999999999999e-05 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678482+0000", 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3318999999999997e-05 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685027+0000", 2024-09-12T07:19:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0065449169999999999 2024-09-12T07:19:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685068+0000", 2024-09-12T07:19:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1328000000000001e-05 2024-09-12T07:19:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:594 3.f e28/24 3:f4a2090e:::rbd_object_map.116011b0bca6:head v 28'18, mlcod=28'14)", 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.678916+0000", 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 10.751230862, 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006184691, 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678916+0000", 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678916+0000", 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678917+0000", 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8500000000000002e-07 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678921+0000", 2024-09-12T07:19:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9330000000000003e-06 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678929+0000", 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9980000000000003e-06 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678933+0000", 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4229999999999998e-06 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678984+0000", 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0831000000000002e-05 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679007+0000", 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3430000000000001e-05 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685070+0000", 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00606238 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685101+0000", 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1211000000000003e-05 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:600 3.f e28/24 3:f4a2090e:::rbd_object_map.116011b0bca6:head v 28'19, mlcod=28'14)", 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.679546+0000", 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 10.750600895, 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059521690000000002, 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679546+0000", 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679546+0000", 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679547+0000", 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9200000000000001e-07 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679552+0000", 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6239999999999999e-06 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679564+0000", 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1704e-05 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679572+0000", 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7170000000000008e-06 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679626+0000", 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3841000000000002e-05 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679639+0000", 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3806e-05 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685471+0000", 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058312909999999997 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685498+0000", 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7594e-05 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:601 3.f e28/24 3:f4a2090e:::rbd_object_map.116011b0bca6:head v 28'20, mlcod=28'14)", 2024-09-12T07:19:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.680100+0000", 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 10.750047295, 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054139380000000001, 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680100+0000", 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680100+0000", 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680100+0000", 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9599999999999999e-07 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680114+0000", 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3614999999999999e-05 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680122+0000", 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1210000000000007e-06 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680126+0000", 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4469999999999999e-06 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680178+0000", 2024-09-12T07:19:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1515999999999999e-05 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.680201+0000", 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.304e-05 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685500+0000", 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005299242 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685514+0000", 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3461e-05 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1331 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'20, mlcod=29'17)", 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.746281+0000", 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.6838659190000005, 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053784150000000001, 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746281+0000", 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746281+0000", 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746282+0000", 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7899999999999998e-07 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746289+0000", 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2690000000000002e-06 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746301+0000", 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1544e-05 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746307+0000", 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4060000000000002e-06 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746356+0000", 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9246999999999998e-05 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.746371+0000", 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5101e-05 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.751626+0000", 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0052545680000000003 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.751660+0000", 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3600999999999999e-05 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1332 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'21, mlcod=29'17)", 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.747073+0000", 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.6830741339999999, 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0076436810000000003, 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747073+0000", 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747073+0000", 2024-09-12T07:19:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747073+0000", 2024-09-12T07:19:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4899999999999995e-07 2024-09-12T07:19:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747077+0000", 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6270000000000002e-06 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747086+0000", 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.302e-06 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747091+0000", 2024-09-12T07:19:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4839999999999998e-06 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747137+0000", 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5896000000000002e-05 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747157+0000", 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-05 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754641+0000", 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0074841810000000003 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754717+0000", 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5562000000000006e-05 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1333 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'22, mlcod=29'17)", 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.747710+0000", 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.6824374090000003, 2024-09-12T07:19:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070469909999999998, 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747710+0000", 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747710+0000", 2024-09-12T07:19:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747710+0000", 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747713+0000", 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1300000000000001e-06 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747720+0000", 2024-09-12T07:19:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7179999999999999e-06 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747723+0000", 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5190000000000001e-06 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747773+0000", 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9549000000000003e-05 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747795+0000", 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2450999999999999e-05 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754722+0000", 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0069261949999999996 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754757+0000", 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4995e-05 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1334 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'23, mlcod=29'18)", 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.748316+0000", 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.6818311989999994, 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0064707979999999998, 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748316+0000", 2024-09-12T07:19:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748316+0000", 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748316+0000", 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748320+0000", 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7699999999999999e-06 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748329+0000", 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4100000000000008e-06 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748333+0000", 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2729999999999996e-06 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748376+0000", 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3464000000000003e-05 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748398+0000", 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1348999999999998e-05 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754758+0000", 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0063606840000000001 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754787+0000", 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8357999999999999e-05 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1335 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'24, mlcod=29'18)", 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.748830+0000", 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.6813165909999999, 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059871769999999998, 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748830+0000", 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748830+0000", 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748831+0000", 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7999999999999995e-07 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748835+0000", 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1139999999999999e-06 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748845+0000", 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4569999999999994e-06 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748849+0000", 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6009999999999997e-06 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748901+0000", 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2280000000000001e-05 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748922+0000", 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0047999999999998e-05 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754788+0000", 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058663999999999999 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754818+0000", 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9697e-05 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1336 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'25, mlcod=29'18)", 2024-09-12T07:19:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.749482+0000", 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.6806651919999993, 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053627120000000004, 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749482+0000", 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749482+0000", 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749483+0000", 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6499999999999999e-07 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749488+0000", 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749500+0000", 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1914e-05 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749508+0000", 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9149999999999996e-06 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749556+0000", 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8766e-05 2024-09-12T07:19:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.749578+0000", 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1500000000000001e-05 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754819+0000", 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0052410199999999999 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754845+0000", 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5752e-05 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1687 3.1c e30/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 30'71, mlcod=30'68)", 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:49.557056+0000", 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.8730914569999992, 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006603757, 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557056+0000", 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557056+0000", 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557056+0000", 2024-09-12T07:19:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0200000000000002e-07 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557062+0000", 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9850000000000003e-06 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557071+0000", 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1770000000000004e-06 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557076+0000", 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4689999999999999e-06 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557117+0000", 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1189999999999997e-05 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557134+0000", 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6637999999999999e-05 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563625+0000", 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0064912679999999997 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563659+0000", 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4428000000000003e-05 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1688 3.1c e30/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 30'72, mlcod=30'68)", 2024-09-12T07:19:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:49.557723+0000", 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.8724239849999993, 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059720429999999998, 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557723+0000", 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557723+0000", 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557724+0000", 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7599999999999996e-07 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557730+0000", 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1290000000000001e-06 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557744+0000", 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3862999999999999e-05 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557751+0000", 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4070000000000004e-06 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557806+0000", 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4379000000000002e-05 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557827+0000", 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1410999999999999e-05 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563662+0000", 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058354230000000002 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563695+0000", 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2654999999999998e-05 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:2067 3.1d e30/24 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:head v 30'14, mlcod=28'12)", 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:49.858445+0000", 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.5717015829999994, 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059962629999999999, 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.858445+0000", 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.858445+0000", 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.858446+0000", 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3600000000000003e-07 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859175+0000", 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00072900199999999997 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859191+0000", 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5376e-05 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859198+0000", 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4989999999999997e-06 2024-09-12T07:19:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859263+0000", 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4519999999999999e-05 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859281+0000", 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7971000000000001e-05 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.864380+0000", 2024-09-12T07:19:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050996720000000004 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.864442+0000", 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1487000000000003e-05 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "pg_update_log_missing(3.9 epoch 34/24 rep_tid 61 entries 34'63 (0'0) error 3:96683ca3:::rbd_data.11603e8faf82.0000000000000059:head by client.4448.0:2907 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'61)", 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.558054+0000", 2024-09-12T07:19:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.8720933029999998, 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054770959999999999, 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.558054+0000", 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.558054+0000", 2024-09-12T07:19:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.558054+0000", 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3e-07 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.558059+0000", 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8489999999999997e-06 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.558075+0000", 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6093999999999999e-05 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.558090+0000", 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4885000000000001e-05 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.558181+0000", 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1633999999999997e-05 2024-09-12T07:19:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.563531+0000", 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053495039999999997 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.816836279, 2024-09-12T07:19:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40167 3.e e37/24 3:738b9f5f:::rbd_data.116063d335b3.000000000000000c:head v 37'71, mlcod=37'68)", 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.741468+0000", 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.68867885299999998, 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056638929999999997, 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.741468+0000", 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.741468+0000", 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.741469+0000", 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4399999999999999e-07 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.741477+0000", 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.191e-06 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.741490+0000", 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2798e-05 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.741497+0000", 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1760000000000004e-06 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.741555+0000", 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7556999999999998e-05 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.741573+0000", 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8505999999999999e-05 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.747106+0000", 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055329569999999998 2024-09-12T07:19:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.747132+0000", 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5863999999999998e-05 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40199 3.11 e37/24 3:8ca86d44:::rbd_data.116063d335b3.0000000000000003:head v 37'27, mlcod=37'25)", 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.769887+0000", 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.66025997400000003, 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056531309999999996, 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769887+0000", 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769887+0000", 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769887+0000", 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769891+0000", 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6169999999999998e-06 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769898+0000", 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9770000000000003e-06 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769903+0000", 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0350000000000001e-06 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769956+0000", 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3248999999999998e-05 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769976+0000", 2024-09-12T07:19:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9218000000000001e-05 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.775507+0000", 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055311249999999996 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.775540+0000", 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3516999999999998e-05 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40196 3.13 e37/24 3:c93134b0:::rbd_data.116063d335b3.0000000000000000:head v 37'51, mlcod=37'49)", 2024-09-12T07:19:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.769942+0000", 2024-09-12T07:19:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.66020533800000003, 2024-09-12T07:19:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056008840000000004, 2024-09-12T07:19:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769942+0000", 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769942+0000", 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769942+0000", 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3600000000000004e-07 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769946+0000", 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6339999999999999e-06 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769953+0000", 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3010000000000001e-06 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.769958+0000", 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2000000000000002e-06 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.770019+0000", 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0303000000000002e-05 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.770037+0000", 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8578000000000001e-05 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.775508+0000", 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054704920000000004 2024-09-12T07:19:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.775543+0000", 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4839999999999998e-05 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40356 3.11 e37/24 3:894561fa:::rbd_data.116063d335b3.0000000000000004:head v 37'30, mlcod=37'28)", 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:58.014063+0000", 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.41608453000000001, 2024-09-12T07:19:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059075530000000003, 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014063+0000", 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014063+0000", 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014063+0000", 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5000000000000002e-07 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014072+0000", 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2989999999999997e-06 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014082+0000", 2024-09-12T07:19:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0000000000000001e-05 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014088+0000", 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3369999999999996e-06 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014138+0000", 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0111000000000002e-05 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014168+0000", 2024-09-12T07:19:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9802e-05 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.019936+0000", 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057680209999999999 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.019970+0000", 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4233000000000002e-05 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40359 3.15 e37/33 3:ac8f5600:::rbd_data.116063d335b3.0000000000000007:head v 37'12, mlcod=37'10)", 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:58.014197+0000", 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.41594956799999999, 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057866949999999997, 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014197+0000", 2024-09-12T07:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014197+0000", 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014198+0000", 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.06e-07 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014203+0000", 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2959999999999998e-06 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014212+0000", 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0489999999999993e-06 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014219+0000", 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4400000000000002e-06 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014282+0000", 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3088000000000003e-05 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014301+0000", 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8950999999999999e-05 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.019943+0000", 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056423050000000002 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.019984+0000", 2024-09-12T07:19:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000001e-05 2024-09-12T07:19:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.519 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:58.520 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:19:58.522 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:58.524+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:58.624 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:58.625+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:58.669 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:19:58.669 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:58.677 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:19:58.724 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:58.725+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:58.784 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:19:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:19:58.784 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:58.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:19:58.825 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:58.826+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:58.925 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:58.927+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:58.941 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:19:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:19:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:19:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:7769 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7452672~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.956783+0000", 2024-09-12T07:19:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.9849240320000003, 2024-09-12T07:19:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052361474999999998, 2024-09-12T07:19:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7769 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956783+0000", 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956783+0000", 2024-09-12T07:19:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956784+0000", 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5900000000000002e-07 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956786+0000", 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.018e-06 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956803+0000", 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7847e-05 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956809+0000", 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8839999999999999e-06 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.008926+0000", 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05211673 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.009021+0000", 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5099999999999994e-05 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.009145+0000", 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000123437 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:7771 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7457792~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.956822+0000", 2024-09-12T07:19:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.9848846260000004, 2024-09-12T07:19:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052560981, 2024-09-12T07:19:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7771 2024-09-12T07:19:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956822+0000", 2024-09-12T07:19:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.956822+0000", 2024-09-12T07:19:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.961941+0000", 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0051181120000000002 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.961946+0000", 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9110000000000001e-06 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.961960+0000", 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4545999999999999e-05 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.961968+0000", 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.009180+0000", 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.047212744000000001 2024-09-12T07:19:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.009274+0000", 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4186999999999998e-05 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.009383+0000", 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109027 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:7819 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7565312~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.962546+0000", 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.9791608639999998, 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052269815999999997, 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7819 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962546+0000", 2024-09-12T07:19:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962546+0000", 2024-09-12T07:19:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962547+0000", 2024-09-12T07:19:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:19:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962549+0000", 2024-09-12T07:19:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2340000000000001e-06 2024-09-12T07:19:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962557+0000", 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3350000000000007e-06 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962564+0000", 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7669999999999998e-06 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.014624+0000", 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052060141999999997 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.014721+0000", 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6410999999999996e-05 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.014816+0000", 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5494000000000004e-05 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:7821 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7570432~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.962573+0000", 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.9791342040000002, 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052477237000000003, 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7821 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962573+0000", 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962573+0000", 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962573+0000", 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3000000000000001e-07 2024-09-12T07:19:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962575+0000", 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1600000000000001e-06 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962584+0000", 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.225e-06 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962589+0000", 2024-09-12T07:19:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.383e-06 2024-09-12T07:19:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.014850+0000", 2024-09-12T07:19:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052260747000000003 2024-09-12T07:19:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.014939+0000", 2024-09-12T07:19:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9600999999999998e-05 2024-09-12T07:19:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015050+0000", 2024-09-12T07:19:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000110691 2024-09-12T07:19:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:7823 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7575552~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.962598+0000", 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.9791091129999998, 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052623057000000001, 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7823 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962598+0000", 2024-09-12T07:19:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962598+0000", 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962598+0000", 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5699999999999998e-07 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962601+0000", 2024-09-12T07:19:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0820000000000001e-06 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962609+0000", 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0749999999999998e-06 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962613+0000", 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2740000000000001e-06 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015086+0000", 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052473335000000003 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015170+0000", 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3386000000000004e-05 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015221+0000", 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1447999999999999e-05 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:7826 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7580672~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:53.962618+0000", 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.9790889969999998, 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052727082000000002, 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7826 2024-09-12T07:19:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962618+0000", 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.962618+0000", 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.967976+0000", 2024-09-12T07:19:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053580490000000001 2024-09-12T07:19:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.967981+0000", 2024-09-12T07:19:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0780000000000003e-06 2024-09-12T07:19:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.967991+0000", 2024-09-12T07:19:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4450000000000002e-06 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.967997+0000", 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6969999999999996e-06 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015255+0000", 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.047257368000000001 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015297+0000", 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2073999999999997e-05 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.015345+0000", 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8371000000000002e-05 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8222 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 8151040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.026213+0000", 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.9154944499999997, 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052383225999999998, 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8222 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026213+0000", 2024-09-12T07:19:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026213+0000", 2024-09-12T07:19:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026213+0000", 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7700000000000004e-07 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026216+0000", 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5830000000000001e-06 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026223+0000", 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4059999999999999e-06 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026229+0000", 2024-09-12T07:19:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5690000000000004e-06 2024-09-12T07:19:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078402+0000", 2024-09-12T07:19:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052172812999999998 2024-09-12T07:19:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078496+0000", 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4393000000000002e-05 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078596+0000", 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9884999999999997e-05 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8223 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 6339584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.026236+0000", 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.915471202, 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052592912999999998, 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8223 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026236+0000", 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026236+0000", 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026317+0000", 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0649999999999995e-05 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026320+0000", 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0709999999999999e-06 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026328+0000", 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2660000000000001e-06 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026334+0000", 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1829999999999999e-06 2024-09-12T07:19:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078630+0000", 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052296294 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078722+0000", 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1138999999999996e-05 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078829+0000", 2024-09-12T07:19:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010731 2024-09-12T07:19:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8224 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 8156160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.026345+0000", 2024-09-12T07:19:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.9153624750000002, 2024-09-12T07:19:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052664641999999998, 2024-09-12T07:19:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8224 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026345+0000", 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.026345+0000", 2024-09-12T07:19:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.031887+0000", 2024-09-12T07:19:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055426449999999997 2024-09-12T07:19:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.031891+0000", 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2080000000000002e-06 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.031902+0000", 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.066e-05 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.031910+0000", 2024-09-12T07:19:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5299999999999999e-06 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078885+0000", 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.046975544000000001 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078943+0000", 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7334999999999999e-05 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.079009+0000", 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6719999999999998e-05 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8392 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 6774784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.052760+0000", 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.8889474340000003, 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052240091000000002, 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8392 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.052760+0000", 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.052760+0000", 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.058011+0000", 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00525181 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.058017+0000", 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0719999999999999e-06 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.058030+0000", 2024-09-12T07:19:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3915e-05 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.058037+0000", 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1090000000000003e-06 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.104765+0000", 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.046728891000000002 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.104879+0000", 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113256 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.105000+0000", 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000121038 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8445 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 6948864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.059945+0000", 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.8817620430000002, 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052372780000000001, 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8445 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059945+0000", 2024-09-12T07:19:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059945+0000", 2024-09-12T07:19:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059946+0000", 2024-09-12T07:19:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2499999999999995e-07 2024-09-12T07:19:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059948+0000", 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7599999999999998e-06 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059956+0000", 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8269999999999998e-06 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059962+0000", 2024-09-12T07:19:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7420000000000003e-06 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.112121+0000", 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052159097000000001 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.112213+0000", 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1531000000000002e-05 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.112318+0000", 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010529799999999999 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8446 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.059970+0000", 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.8817373340000003, 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052538337999999997, 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8446 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059970+0000", 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.059970+0000", 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066075+0000", 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061051339999999999 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066079+0000", 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7950000000000001e-06 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066089+0000", 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0078000000000001e-05 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066094+0000", 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0679999999999996e-06 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.112345+0000", 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.046251249000000001 2024-09-12T07:19:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.112420+0000", 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5246000000000004e-05 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.112508+0000", 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7768000000000004e-05 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8477 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7030784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.066789+0000", 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.8749181699999999, 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052337458000000003, 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8477 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066789+0000", 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066789+0000", 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066789+0000", 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066792+0000", 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4219999999999999e-06 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066799+0000", 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7100000000000001e-06 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066803+0000", 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9080000000000003e-06 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.118930+0000", 2024-09-12T07:19:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052126611000000003 2024-09-12T07:19:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119023+0000", 2024-09-12T07:19:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2851999999999998e-05 2024-09-12T07:19:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119126+0000", 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010345599999999999 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8478 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7035904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.066809+0000", 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.8748984609999999, 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052560395000000003, 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8478 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066809+0000", 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066809+0000", 2024-09-12T07:19:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066809+0000", 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8999999999999998e-07 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066810+0000", 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5179999999999999e-06 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066814+0000", 2024-09-12T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8929999999999998e-06 2024-09-12T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066817+0000", 2024-09-12T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6579999999999998e-06 2024-09-12T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119172+0000", 2024-09-12T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05235542 2024-09-12T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119261+0000", 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8315999999999999e-05 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119369+0000", 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0001083 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8479 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7426048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.066820+0000", 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.8748866629999998, 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052787775000000002, 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8479 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066820+0000", 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.066820+0000", 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.071929+0000", 2024-09-12T07:19:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005108334 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.071934+0000", 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0220000000000004e-06 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.071953+0000", 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8811e-05 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.071962+0000", 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3570000000000004e-06 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119414+0000", 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.047451568999999999 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119505+0000", 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1061000000000001e-05 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.119608+0000", 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000103621 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8502 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7482368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.072395+0000", 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.8693117030000002, 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052348634999999998, 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8502 2024-09-12T07:19:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072395+0000", 2024-09-12T07:19:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072395+0000", 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072396+0000", 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9500000000000002e-07 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072399+0000", 2024-09-12T07:19:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072407+0000", 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.96e-06 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072413+0000", 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.246e-06 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.124598+0000", 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052185441999999999 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.124665+0000", 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6960999999999998e-05 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.124744+0000", 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8786000000000001e-05 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8503 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7102464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.072422+0000", 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.8692855289999999, 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052572356000000001, 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8503 2024-09-12T07:19:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072422+0000", 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072422+0000", 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072422+0000", 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3300000000000002e-07 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072424+0000", 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3209999999999999e-06 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072433+0000", 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3410000000000003e-06 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072438+0000", 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7039999999999999e-06 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.124783+0000", 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052344684000000002 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.124880+0000", 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7226999999999996e-05 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.124994+0000", 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113546 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8504 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7487488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.072446+0000", 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.8692614150000004, 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052688156, 2024-09-12T07:19:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8504 2024-09-12T07:19:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072446+0000", 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072446+0000", 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072446+0000", 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6499999999999999e-07 2024-09-12T07:19:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072448+0000", 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1320000000000001e-06 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072457+0000", 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9590000000000001e-06 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072463+0000", 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4739999999999996e-06 2024-09-12T07:19:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125031+0000", 2024-09-12T07:19:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052567908000000003 2024-09-12T07:19:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125076+0000", 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5569999999999999e-05 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125134+0000", 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7648000000000001e-05 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8505 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7107584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.072470+0000", 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.8692370279999997, 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052904026, 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8505 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072470+0000", 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072470+0000", 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072471+0000", 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5400000000000002e-07 2024-09-12T07:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072473+0000", 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1880000000000001e-06 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072480+0000", 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2459999999999997e-06 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072487+0000", 2024-09-12T07:19:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1189999999999999e-06 2024-09-12T07:19:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125176+0000", 2024-09-12T07:19:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052689034000000003 2024-09-12T07:19:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125271+0000", 2024-09-12T07:19:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4590000000000001e-05 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125374+0000", 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000103395 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:8506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 7492608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:54.072494+0000", 2024-09-12T07:19:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.8692127369999998, 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.053008686999999999, 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 8506 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072494+0000", 2024-09-12T07:19:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.072494+0000", 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078003+0000", 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055090909999999998 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078008+0000", 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2899999999999996e-06 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078018+0000", 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.006e-05 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.078025+0000", 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7560000000000002e-06 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125404+0000", 2024-09-12T07:19:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.047379457999999999 2024-09-12T07:19:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125448+0000", 2024-09-12T07:19:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3628999999999999e-05 2024-09-12T07:19:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:54.125503+0000", 2024-09-12T07:19:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5402999999999998e-05 2024-09-12T07:19:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.035 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:59.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:19:59.037 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:19:59.038+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:19:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.1 to 1 (10000) 2024-09-12T07:19:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:19:59.137 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:19:59.139+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:19:59.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-12T07:19:59.187 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:19:59.187 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:59.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:19:59.227 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone5 from 0xc0f3d2c (overlap 0x1badf78) to 0x0 2024-09-12T07:19:59.239 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:19:59.241+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:19:59.308 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:19:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:19:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:19:59.309 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:59.317 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:19:59.342 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-12T07:19:59.358 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:19:59.360+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:19:59.440 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:59.442+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:59.457 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:19:59.526 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:19:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:19:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:19:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:19:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:19:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 6.9972276640000004, 2024-09-12T07:19:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:19:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:19:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:19:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:19:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:19:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:19:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:19:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:19:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:19:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:19:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:19:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:19:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:19:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:19:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:19:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:19:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:19:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:19:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:19:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:19:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:19:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:19:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:19:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:19:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.145429998, 2024-09-12T07:19:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:19:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:19:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:19:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:19:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:19:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:19:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:19:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:19:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:19:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:19:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:19:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:19:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:19:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:19:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:19:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:19:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:19:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:19:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:19:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.145306721, 2024-09-12T07:19:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:19:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:19:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:19:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:19:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:19:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:19:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:19:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:19:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:19:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:19:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:19:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:19:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.14510333, 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:19:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.144967503, 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:19:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:19:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:19:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:19:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.14464084699999999, 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:19:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:19:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:19:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:19:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:19:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:19:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:19:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.144601959, 2024-09-12T07:19:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:19:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:19:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:19:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:19:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:19:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:19:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:19:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:19:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:19:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:19:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:19:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:19:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:19:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:19:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:19:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.14423508500000001, 2024-09-12T07:19:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:19:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:19:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:19:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:19:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:19:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:19:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:19:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:19:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:19:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:19:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:19:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:19:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:19:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:19:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:19:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.144100229, 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:19:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.14373033800000001, 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:19:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:19:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:19:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:19:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:19:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:19:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:19:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.14354014000000001, 2024-09-12T07:19:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:19:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:19:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.143320061, 2024-09-12T07:19:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:19:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:19:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:19:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:19:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:19:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:19:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:19:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.14317639600000001, 2024-09-12T07:19:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:19:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:19:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:19:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:19:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:19:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:19:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:19:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:19:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:19:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:19:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:19:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.14295022399999999, 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:19:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:19:59.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:19:59.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:19:59.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:19:59.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:19:59.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:19:59.610 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.142749866, 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.611 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:19:59.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.612 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:19:59.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:19:59.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:19:59.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.14234198300000001, 2024-09-12T07:19:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:19:59.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:19:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:19:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:19:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:19:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:19:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.141876101, 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.620 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:19:59.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:19:59.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:19:59.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:19:59.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:19:59.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:19:59.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:19:59.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:19:59.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:19:59.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:19:59.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:19:59.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:19:59.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:19:59.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.14027389100000001, 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.625 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.629 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.139739856, 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:19:59.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:19:59.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:19:59.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:19:59.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:19:59.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42452 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 484352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.393519+0000", 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.13341715000000001, 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055174381000000002, 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.634 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42452 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.393519+0000", 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.393519+0000", 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.398882+0000", 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0053626469999999999 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.398887+0000", 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1100000000000002e-06 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.398899+0000", 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1846e-05 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.398907+0000", 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.448527+0000", 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.049619784 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.448609+0000", 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1978999999999996e-05 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.448694+0000", 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5243999999999999e-05 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.638 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:19:59.640 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:19:59.640 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:19:59.641+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:19:59.668 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:19:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:19:59.698 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:19:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:19:59.723 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:19:59.724 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:19:59.740 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:19:59.741+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:19:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:19:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:19:59.773 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:19:59.773 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:19:59.792 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:19:59.792 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:19:59.814 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:19:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:19:59.832 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:19:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:19:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48141 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.789777+0000", 2024-09-12T07:19:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.028566212000000001, 2024-09-12T07:19:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028585470000000002, 2024-09-12T07:19:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48141 2024-09-12T07:19:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789777+0000", 2024-09-12T07:19:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789777+0000", 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789778+0000", 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4300000000000003e-07 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789780+0000", 2024-09-12T07:19:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2869999999999999e-06 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789787+0000", 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7059999999999998e-06 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789792+0000", 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0660000000000003e-06 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48191 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.790805+0000", 2024-09-12T07:19:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027538407000000001, 2024-09-12T07:19:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027614857, 2024-09-12T07:19:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48191 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790805+0000", 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790805+0000", 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790806+0000", 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3600000000000004e-07 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790808+0000", 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3300000000000001e-06 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790822+0000", 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3804e-05 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790828+0000", 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2299999999999996e-06 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48258 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791957+0000", 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026386641999999998, 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026513848999999999, 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48258 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791957+0000", 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791957+0000", 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791957+0000", 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791959+0000", 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0530000000000001e-06 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791966+0000", 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.793e-06 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791973+0000", 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2699999999999999e-06 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48318 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792806+0000", 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025537318, 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025715545999999999, 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48318 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792806+0000", 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792806+0000", 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792807+0000", 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4999999999999998e-07 2024-09-12T07:19:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792809+0000", 2024-09-12T07:19:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0789999999999999e-06 2024-09-12T07:19:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792816+0000", 2024-09-12T07:19:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9800000000000001e-06 2024-09-12T07:19:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792821+0000", 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3120000000000001e-06 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48380 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793669+0000", 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024674772000000001, 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024899270000000001, 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48380 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793669+0000", 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793669+0000", 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793669+0000", 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5200000000000002e-07 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793672+0000", 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4590000000000001e-06 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793679+0000", 2024-09-12T07:19:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2450000000000001e-06 2024-09-12T07:19:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793684+0000", 2024-09-12T07:19:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3009999999999997e-06 2024-09-12T07:19:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48445 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801143+0000", 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017200409999999999, 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017451583999999999, 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48445 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801143+0000", 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801143+0000", 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801144+0000", 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2600000000000002e-07 2024-09-12T07:19:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801146+0000", 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4109999999999999e-06 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801153+0000", 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2099999999999996e-06 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801159+0000", 2024-09-12T07:19:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6160000000000001e-06 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48512 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807885+0000", 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010458623, 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010733618, 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48512 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807885+0000", 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807885+0000", 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807885+0000", 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7199999999999999e-07 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807888+0000", 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7889999999999999e-06 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807895+0000", 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0979999999999996e-06 2024-09-12T07:19:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807901+0000", 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5840000000000002e-06 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48142 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.789798+0000", 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.028544915000000001, 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028843852999999999, 2024-09-12T07:19:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48142 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789798+0000", 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789798+0000", 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789799+0000", 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7700000000000005e-07 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789801+0000", 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1430000000000001e-06 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789814+0000", 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3264999999999999e-05 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789820+0000", 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8039999999999998e-06 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48193 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.790836+0000", 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027507317, 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027829858999999998, 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48193 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790836+0000", 2024-09-12T07:19:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790836+0000", 2024-09-12T07:19:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790837+0000", 2024-09-12T07:19:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8599999999999998e-07 2024-09-12T07:19:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790839+0000", 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4320000000000002e-06 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790846+0000", 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.114e-06 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790852+0000", 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4369999999999998e-06 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48260 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791991+0000", 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026352322000000001, 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026699572000000001, 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48260 2024-09-12T07:19:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791991+0000", 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791991+0000", 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791992+0000", 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8999999999999996e-07 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791994+0000", 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4080000000000001e-06 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792002+0000", 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5469999999999999e-06 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792007+0000", 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8529999999999997e-06 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48321 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792828+0000", 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025515156000000001, 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025885893, 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48321 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792828+0000", 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792828+0000", 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792829+0000", 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6100000000000002e-07 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792831+0000", 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.916e-06 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792837+0000", 2024-09-12T07:19:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3790000000000003e-06 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792843+0000", 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2369999999999997e-06 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48381 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793691+0000", 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024652256000000001, 2024-09-12T07:19:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025047186999999999, 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48381 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793691+0000", 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793691+0000", 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793692+0000", 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793693+0000", 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.888e-06 2024-09-12T07:19:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793703+0000", 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1309999999999995e-06 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793708+0000", 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2340000000000002e-06 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48447 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801166+0000", 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017176903, 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017595746999999998, 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48447 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801166+0000", 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801166+0000", 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801167+0000", 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4000000000000002e-07 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801169+0000", 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0150000000000002e-06 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801176+0000", 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1369999999999996e-06 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801181+0000", 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4160000000000003e-06 2024-09-12T07:19:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48514 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807909+0000", 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010434439, 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01087686, 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48514 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807909+0000", 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807909+0000", 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807909+0000", 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4000000000000002e-07 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807912+0000", 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.605e-06 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807919+0000", 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8940000000000004e-06 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807924+0000", 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3090000000000003e-06 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48143 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.789828+0000", 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.02851501, 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028998024000000001, 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48143 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789828+0000", 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789828+0000", 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789829+0000", 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.51e-07 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789831+0000", 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3269999999999999e-06 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789883+0000", 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2122e-05 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789890+0000", 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3090000000000001e-06 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48195 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.790859+0000", 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027484399999999999, 2024-09-12T07:19:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028025359, 2024-09-12T07:19:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48195 2024-09-12T07:19:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790859+0000", 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790859+0000", 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790859+0000", 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7399999999999998e-07 2024-09-12T07:19:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790862+0000", 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0940000000000002e-06 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790877+0000", 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5928999999999999e-05 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790883+0000", 2024-09-12T07:19:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.896e-06 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48262 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792016+0000", 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026327234000000001, 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026916881, 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48262 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792016+0000", 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792016+0000", 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792017+0000", 2024-09-12T07:19:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8100000000000003e-07 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792019+0000", 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8449999999999999e-06 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792026+0000", 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6379999999999998e-06 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792031+0000", 2024-09-12T07:19:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48323 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792852+0000", 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025491570000000002, 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026133940000000001, 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48323 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792852+0000", 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792852+0000", 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792852+0000", 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6600000000000002e-07 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792855+0000", 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2709999999999999e-06 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792862+0000", 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0840000000000003e-06 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792874+0000", 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.253e-05 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48383 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793715+0000", 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024628473000000001, 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025320077999999999, 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48383 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793715+0000", 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793715+0000", 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793715+0000", 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4700000000000001e-07 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793722+0000", 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7850000000000003e-06 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793729+0000", 2024-09-12T07:19:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1450000000000002e-06 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793735+0000", 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5639999999999996e-06 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48449 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801189+0000", 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017154157, 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017894501, 2024-09-12T07:19:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48449 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801189+0000", 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801189+0000", 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801190+0000", 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4299999999999998e-07 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801192+0000", 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0710000000000002e-06 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801199+0000", 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9700000000000002e-06 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801214+0000", 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.554e-05 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48516 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807952+0000", 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.01039124, 2024-09-12T07:19:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011180924, 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48516 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807952+0000", 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807952+0000", 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807953+0000", 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.75e-07 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807956+0000", 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9059999999999998e-06 2024-09-12T07:19:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807964+0000", 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5769999999999994e-06 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807970+0000", 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8150000000000002e-06 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48144 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.789900+0000", 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.028443743, 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029282426, 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48144 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789900+0000", 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789900+0000", 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789900+0000", 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9200000000000001e-07 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789903+0000", 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0010000000000002e-06 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789911+0000", 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4619999999999999e-06 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789916+0000", 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.434e-06 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48197 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.790892+0000", 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.02745125, 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028339751999999999, 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48197 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790892+0000", 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790892+0000", 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790893+0000", 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6899999999999997e-07 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790900+0000", 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.61e-06 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790908+0000", 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7689999999999996e-06 2024-09-12T07:19:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790914+0000", 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6930000000000004e-06 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48264 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792039+0000", 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026304722999999999, 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027241766000000001, 2024-09-12T07:19:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48264 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792039+0000", 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792039+0000", 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792039+0000", 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792041+0000", 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9609999999999999e-06 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792048+0000", 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6850000000000004e-06 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792053+0000", 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2909999999999998e-06 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48325 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792883+0000", 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.02546031, 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026445682000000002, 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48325 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792883+0000", 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792883+0000", 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792884+0000", 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0299999999999999e-07 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792886+0000", 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5459999999999998e-06 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792894+0000", 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.746e-06 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792899+0000", 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5960000000000003e-06 2024-09-12T07:19:59.914 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48385 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793752+0000", 2024-09-12T07:19:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024591544999999999, 2024-09-12T07:19:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025624503, 2024-09-12T07:19:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.915 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.916 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.916 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.916 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48385 2024-09-12T07:19:59.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.916 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.916 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793752+0000", 2024-09-12T07:19:59.916 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793752+0000", 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793752+0000", 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6100000000000001e-07 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793755+0000", 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3609999999999999e-06 2024-09-12T07:19:59.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793763+0000", 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0560000000000005e-06 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793769+0000", 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7300000000000002e-06 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.918 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48452 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801222+0000", 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017120954000000001, 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018204125000000002, 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48452 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801222+0000", 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801222+0000", 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801223+0000", 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2799999999999996e-07 2024-09-12T07:19:59.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801225+0000", 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.384e-06 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801233+0000", 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4359999999999996e-06 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801238+0000", 2024-09-12T07:19:59.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4970000000000001e-06 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48518 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807979+0000", 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010364104000000001, 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011495366, 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.922 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48518 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807979+0000", 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807979+0000", 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.924 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.924 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807980+0000", 2024-09-12T07:19:59.924 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2e-07 2024-09-12T07:19:59.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.924 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.924 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.924 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807982+0000", 2024-09-12T07:19:59.924 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2089999999999999e-06 2024-09-12T07:19:59.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.924 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.924 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.924 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807989+0000", 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.533e-06 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807996+0000", 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9920000000000001e-06 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48145 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.925 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.789923+0000", 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.02841991, 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029600501000000001, 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48145 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789923+0000", 2024-09-12T07:19:59.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789923+0000", 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789924+0000", 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6899999999999998e-07 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789926+0000", 2024-09-12T07:19:59.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789933+0000", 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4640000000000003e-06 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789938+0000", 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3639999999999998e-06 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48199 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.790922+0000", 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027421187, 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028649696999999998, 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48199 2024-09-12T07:19:59.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790922+0000", 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790922+0000", 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790923+0000", 2024-09-12T07:19:59.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0900000000000002e-07 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790925+0000", 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5100000000000001e-06 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790933+0000", 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.277e-06 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790938+0000", 2024-09-12T07:19:59.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2179999999999998e-06 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48266 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792060+0000", 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026283530999999999, 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027561367, 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.932 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48266 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792060+0000", 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792060+0000", 2024-09-12T07:19:59.933 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792060+0000", 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792062+0000", 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.934 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792069+0000", 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9439999999999999e-06 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792074+0000", 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-06 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48327 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792908+0000", 2024-09-12T07:19:59.935 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025435867000000001, 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026761572000000001, 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48327 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792908+0000", 2024-09-12T07:19:59.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792908+0000", 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792908+0000", 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7099999999999997e-07 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.937 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792910+0000", 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.108e-06 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792917+0000", 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.827e-06 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792924+0000", 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8449999999999997e-06 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.938 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48389 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793778+0000", 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024564954999999999, 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025938955999999999, 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.939 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48389 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793778+0000", 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793778+0000", 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793779+0000", 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4999999999999998e-07 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793781+0000", 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0169999999999999e-06 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793788+0000", 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2629999999999997e-06 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793794+0000", 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4940000000000003e-06 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48454 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801330+0000", 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017013818, 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018436099000000001, 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.942 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48454 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801330+0000", 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801330+0000", 2024-09-12T07:19:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801330+0000", 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7800000000000001e-07 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801333+0000", 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0299999999999998e-06 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801344+0000", 2024-09-12T07:19:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0332999999999999e-05 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801350+0000", 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5699999999999998e-06 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48520 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.808003+0000", 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010339931, 2024-09-12T07:19:59.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011857006, 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48520 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808003+0000", 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808003+0000", 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808004+0000", 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808006+0000", 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5950000000000001e-06 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808013+0000", 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6039999999999998e-06 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808018+0000", 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0200000000000002e-06 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48146 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.948 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.789945+0000", 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.028398244999999999, 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029972108000000001, 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48146 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789945+0000", 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789945+0000", 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789946+0000", 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789948+0000", 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.928e-06 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789954+0000", 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6810000000000001e-06 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789959+0000", 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.215e-06 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48201 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.790946+0000", 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027397527000000001, 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029023576999999998, 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48201 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.952 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790946+0000", 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790946+0000", 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790946+0000", 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5600000000000001e-07 2024-09-12T07:19:59.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790949+0000", 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2620000000000001e-06 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.790994+0000", 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5015999999999997e-05 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791010+0000", 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6648999999999999e-05 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48268 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792082+0000", 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026261385000000002, 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027936231999999998, 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.955 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48268 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792082+0000", 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792082+0000", 2024-09-12T07:19:59.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792082+0000", 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3000000000000001e-07 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792085+0000", 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.243e-06 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792092+0000", 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3010000000000001e-06 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792098+0000", 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6509999999999998e-06 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48329 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792931+0000", 2024-09-12T07:19:59.958 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025412427000000001, 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027135412000000001, 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48329 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792931+0000", 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792931+0000", 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792931+0000", 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792933+0000", 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9870000000000002e-06 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792940+0000", 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792946+0000", 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4060000000000004e-06 2024-09-12T07:19:59.961 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48391 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793801+0000", 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024542042, 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026317376, 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.962 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48391 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793801+0000", 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793801+0000", 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793802+0000", 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8799999999999998e-07 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793804+0000", 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.973e-06 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793811+0000", 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1350000000000003e-06 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793816+0000", 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2519999999999999e-06 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48456 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801360+0000", 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.016983080000000001, 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018806363999999999, 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48456 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801360+0000", 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801360+0000", 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801361+0000", 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.51e-07 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801363+0000", 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4279999999999999e-06 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801371+0000", 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4800000000000004e-06 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801376+0000", 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5790000000000003e-06 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.968 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48522 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.808026+0000", 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010317745999999999, 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012194543, 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48522 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808026+0000", 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808026+0000", 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808026+0000", 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.32e-07 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808028+0000", 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.227e-06 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808036+0000", 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4429999999999997e-06 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808041+0000", 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999999e-06 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.971 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48147 3.13 3:c94a2238:::rbd_data.1160947fa40f.0000000000000003:6 [read 1560576~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.789966+0000", 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.028376908999999999, 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030310172, 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48147 2024-09-12T07:19:59.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789966+0000", 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789966+0000", 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789967+0000", 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789969+0000", 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.995e-06 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789976+0000", 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6069999999999996e-06 2024-09-12T07:19:59.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.789981+0000", 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1019999999999996e-06 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.820218+0000", 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030237362 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:19:59.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.820259+0000", 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1275e-05 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48203 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791022+0000", 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027321872000000001, 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029317006999999999, 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.976 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48203 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791022+0000", 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791022+0000", 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791022+0000", 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7299999999999996e-07 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791025+0000", 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0630000000000002e-06 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791033+0000", 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7510000000000008e-06 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791039+0000", 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6840000000000001e-06 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48270 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.979 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792104+0000", 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026238945, 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028281923, 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48270 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792104+0000", 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792104+0000", 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792105+0000", 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792107+0000", 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.013e-06 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792113+0000", 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4509999999999997e-06 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792118+0000", 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9799999999999998e-06 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.982 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48331 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792954+0000", 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025389418, 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027485368, 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48331 2024-09-12T07:19:59.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792954+0000", 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792954+0000", 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792954+0000", 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0999999999999999e-07 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792956+0000", 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0710000000000002e-06 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792971+0000", 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4209999999999999e-05 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792977+0000", 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3500000000000002e-06 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48393 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793824+0000", 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024519203999999999, 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026663276999999999, 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.986 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48393 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793824+0000", 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793824+0000", 2024-09-12T07:19:59.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793825+0000", 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0999999999999999e-07 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793827+0000", 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.979e-06 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793834+0000", 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9809999999999997e-06 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793839+0000", 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2850000000000002e-06 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48458 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.989 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801384+0000", 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.016959345000000001, 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019154551999999998, 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48458 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801384+0000", 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801384+0000", 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801384+0000", 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9499999999999998e-07 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801391+0000", 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.764e-06 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801399+0000", 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.396e-06 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801405+0000", 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3219999999999998e-06 2024-09-12T07:19:59.992 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48524 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.808048+0000", 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010295604, 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012539544, 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.993 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48524 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808048+0000", 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808048+0000", 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808048+0000", 2024-09-12T07:19:59.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2800000000000002e-07 2024-09-12T07:19:59.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808050+0000", 2024-09-12T07:19:59.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.255e-06 2024-09-12T07:19:59.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808057+0000", 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808061+0000", 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7940000000000002e-06 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48205 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791046+0000", 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027297161, 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029596345999999999, 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48205 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:19:59.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791046+0000", 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791046+0000", 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791047+0000", 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.58e-07 2024-09-12T07:19:59.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791049+0000", 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1509999999999998e-06 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791062+0000", 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3517e-05 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:19:59.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791068+0000", 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.993e-06 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48272 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792125+0000", 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026218123999999999, 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028566016, 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.000 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48272 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792125+0000", 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792125+0000", 2024-09-12T07:20:00.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792126+0000", 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.51e-07 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792133+0000", 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8079999999999999e-06 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792140+0000", 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.199e-06 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792145+0000", 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3059999999999997e-06 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48332 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792985+0000", 2024-09-12T07:20:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025358279000000001, 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027759255, 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48332 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792985+0000", 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792985+0000", 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792986+0000", 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9999999999999998e-07 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792988+0000", 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2069999999999998e-06 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792998+0000", 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9210000000000004e-06 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793004+0000", 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.891e-06 2024-09-12T07:20:00.006 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48395 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793859+0000", 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024484586999999999, 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026934493, 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.007 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48395 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793859+0000", 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793859+0000", 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793859+0000", 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.63e-07 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793862+0000", 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3350000000000001e-06 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793875+0000", 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3395e-05 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793882+0000", 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.469e-06 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48460 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.010 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801413+0000", 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.016930739, 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019433063, 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48460 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801413+0000", 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801413+0000", 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801413+0000", 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801415+0000", 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1490000000000001e-06 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801422+0000", 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6019999999999996e-06 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801427+0000", 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1560000000000003e-06 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.013 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48525 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.808069+0000", 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010274828, 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01283217, 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48525 2024-09-12T07:20:00.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808069+0000", 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808069+0000", 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808069+0000", 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5199999999999998e-07 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808071+0000", 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1330000000000002e-06 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808078+0000", 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.635e-06 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808083+0000", 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.933e-06 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48209 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791077+0000", 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027266103999999999, 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029878165000000002, 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.017 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48209 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791077+0000", 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791077+0000", 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791078+0000", 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7899999999999998e-07 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791080+0000", 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5100000000000001e-06 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791088+0000", 2024-09-12T07:20:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2930000000000003e-06 2024-09-12T07:20:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791093+0000", 2024-09-12T07:20:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3410000000000002e-06 2024-09-12T07:20:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.021 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48274 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.021 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792153+0000", 2024-09-12T07:20:00.021 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.02619026, 2024-09-12T07:20:00.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028854178000000001, 2024-09-12T07:20:00.021 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.021 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.021 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.021 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.021 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.021 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.022 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48274 2024-09-12T07:20:00.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.022 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792153+0000", 2024-09-12T07:20:00.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792153+0000", 2024-09-12T07:20:00.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792154+0000", 2024-09-12T07:20:00.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5700000000000002e-07 2024-09-12T07:20:00.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792156+0000", 2024-09-12T07:20:00.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3070000000000001e-06 2024-09-12T07:20:00.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792163+0000", 2024-09-12T07:20:00.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9650000000000002e-06 2024-09-12T07:20:00.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792168+0000", 2024-09-12T07:20:00.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.468e-06 2024-09-12T07:20:00.024 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.024 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48334 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793012+0000", 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025331890999999999, 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028045982000000001, 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.025 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48334 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793012+0000", 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793012+0000", 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793012+0000", 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7500000000000001e-07 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793014+0000", 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1040000000000001e-06 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793021+0000", 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8299999999999998e-06 2024-09-12T07:20:00.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793026+0000", 2024-09-12T07:20:00.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6609999999999997e-06 2024-09-12T07:20:00.028 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.028 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.028 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.028 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48397 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.028 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793892+0000", 2024-09-12T07:20:00.028 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024451578000000002, 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027220267999999999, 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48397 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793892+0000", 2024-09-12T07:20:00.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793892+0000", 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793892+0000", 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793895+0000", 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8059999999999999e-06 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793903+0000", 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5449999999999998e-06 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793908+0000", 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7969999999999997e-06 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.031 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48462 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801512+0000", 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.016831828, 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019648632999999999, 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48462 2024-09-12T07:20:00.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.033 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801512+0000", 2024-09-12T07:20:00.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801512+0000", 2024-09-12T07:20:00.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801512+0000", 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0900000000000001e-07 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801516+0000", 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5659999999999998e-06 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801529+0000", 2024-09-12T07:20:00.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2821e-05 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801533+0000", 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2729999999999996e-06 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48527 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.808089+0000", 2024-09-12T07:20:00.035 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010254236999999999, 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013123266, 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48527 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808089+0000", 2024-09-12T07:20:00.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808089+0000", 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808089+0000", 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1199999999999999e-07 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808092+0000", 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.266e-06 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808098+0000", 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808104+0000", 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1149999999999999e-06 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.038 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48211 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791100+0000", 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027243137000000001, 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030161001999999999, 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.039 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48211 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791100+0000", 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791100+0000", 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791101+0000", 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6100000000000001e-07 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791103+0000", 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0700000000000001e-06 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791112+0000", 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0059999999999998e-06 2024-09-12T07:20:00.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791117+0000", 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3900000000000001e-06 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48276 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792175+0000", 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026167915, 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029138276000000001, 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.042 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48276 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792175+0000", 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792175+0000", 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792176+0000", 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5400000000000002e-07 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792178+0000", 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792184+0000", 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3559999999999998e-06 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792189+0000", 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1599999999999997e-06 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48336 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793034+0000", 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025308944, 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028328346000000001, 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48336 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793034+0000", 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793034+0000", 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793035+0000", 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:20:00.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793037+0000", 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2139999999999999e-06 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793044+0000", 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4729999999999997e-06 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793049+0000", 2024-09-12T07:20:00.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.31e-06 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48399 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793916+0000", 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024427193, 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027500317999999999, 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.049 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48399 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793916+0000", 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793916+0000", 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.794202+0000", 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028554399999999999 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.794208+0000", 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1739999999999997e-06 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.794219+0000", 2024-09-12T07:20:00.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0764000000000001e-05 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.794234+0000", 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5758999999999999e-05 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48464 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801540+0000", 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.016803011, 2024-09-12T07:20:00.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019924999999999998, 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48464 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801540+0000", 2024-09-12T07:20:00.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801540+0000", 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801541+0000", 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9000000000000002e-07 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801542+0000", 2024-09-12T07:20:00.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.57e-06 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801548+0000", 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4559999999999999e-06 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801551+0000", 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.976e-06 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.055 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48528 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.808112+0000", 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010231488, 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013405945000000001, 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48528 2024-09-12T07:20:00.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808112+0000", 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808112+0000", 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808112+0000", 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9099999999999999e-07 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808114+0000", 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0779999999999998e-06 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808121+0000", 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3260000000000001e-06 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808126+0000", 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.147e-06 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48213 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791126+0000", 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027217623, 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030442144000000001, 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.059 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48213 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791126+0000", 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791126+0000", 2024-09-12T07:20:00.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791126+0000", 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9100000000000004e-07 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791129+0000", 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3779999999999999e-06 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791136+0000", 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2960000000000001e-06 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791142+0000", 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5809999999999996e-06 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48278 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.062 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792196+0000", 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026146936999999999, 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029497283999999999, 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48278 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792196+0000", 2024-09-12T07:20:00.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792196+0000", 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792197+0000", 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0200000000000003e-07 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792199+0000", 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0999999999999998e-06 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792206+0000", 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7039999999999997e-06 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792211+0000", 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0969999999999996e-06 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.065 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48338 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793057+0000", 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025286769000000001, 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028687444999999999, 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.066 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48338 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793057+0000", 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793057+0000", 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793057+0000", 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2800000000000002e-07 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793059+0000", 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.12e-06 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793070+0000", 2024-09-12T07:20:00.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.102e-05 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793076+0000", 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4809999999999997e-06 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48403 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.794247+0000", 2024-09-12T07:20:00.069 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024096635000000002, 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027548956999999999, 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48403 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.794247+0000", 2024-09-12T07:20:00.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.794247+0000", 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.794247+0000", 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.75e-07 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.794251+0000", 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.129e-06 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.794258+0000", 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9079999999999995e-06 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.794270+0000", 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1066999999999999e-05 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.072 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48466 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801555+0000", 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.016788608, 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020290404000000001, 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.073 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48466 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801555+0000", 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801555+0000", 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801555+0000", 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3699999999999999e-07 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801556+0000", 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.201e-06 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801560+0000", 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4470000000000002e-06 2024-09-12T07:20:00.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801564+0000", 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9419999999999997e-06 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48530 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.808165+0000", 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010177988000000001, 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013738333, 2024-09-12T07:20:00.076 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48530 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808165+0000", 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808165+0000", 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808166+0000", 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9400000000000005e-07 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808173+0000", 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2470000000000002e-06 2024-09-12T07:20:00.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808182+0000", 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5879999999999998e-06 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808188+0000", 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2400000000000004e-06 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48215 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791149+0000", 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027194644, 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030805299000000001, 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48215 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791149+0000", 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791149+0000", 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791149+0000", 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:20:00.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791151+0000", 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.874e-06 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791191+0000", 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9572e-05 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791208+0000", 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7082e-05 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.082 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48280 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792218+0000", 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026125863999999999, 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029787173, 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48280 2024-09-12T07:20:00.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792218+0000", 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792218+0000", 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792218+0000", 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9000000000000002e-07 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792220+0000", 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9209999999999999e-06 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792227+0000", 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5279999999999998e-06 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792233+0000", 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2970000000000003e-06 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48340 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793087+0000", 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025256104000000001, 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028968872, 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.086 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48340 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793087+0000", 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793087+0000", 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793088+0000", 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793090+0000", 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3760000000000002e-06 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793098+0000", 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5989999999999996e-06 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793104+0000", 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6189999999999996e-06 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48405 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.794278+0000", 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024065014999999999, 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027825616000000001, 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.089 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48405 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.794278+0000", 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.794278+0000", 2024-09-12T07:20:00.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800041+0000", 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057628710000000001 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800046+0000", 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2699999999999998e-06 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800057+0000", 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1430999999999999e-05 2024-09-12T07:20:00.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800065+0000", 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6210000000000004e-06 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48468 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801567+0000", 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.016775993999999999, 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.020588904000000002, 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48468 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801567+0000", 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801567+0000", 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801568+0000", 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.23e-07 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801569+0000", 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.17e-06 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801572+0000", 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2820000000000001e-06 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801575+0000", 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.779e-06 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48532 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.808197+0000", 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010146347999999999, 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014008367000000001, 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48532 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808197+0000", 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808197+0000", 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808197+0000", 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9000000000000002e-07 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808200+0000", 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3700000000000002e-06 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808206+0000", 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6789999999999999e-06 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808211+0000", 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48217 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791224+0000", 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027119465999999998, 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031033944000000001, 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48217 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791224+0000", 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791224+0000", 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791225+0000", 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7400000000000003e-07 2024-09-12T07:20:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791227+0000", 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9469999999999999e-06 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791243+0000", 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6008999999999999e-05 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791250+0000", 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2480000000000001e-06 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.101 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.102 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.103 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48282 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.103 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792239+0000", 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026103997, 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03006758, 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48282 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792239+0000", 2024-09-12T07:20:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792239+0000", 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792240+0000", 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792242+0000", 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9819999999999998e-06 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792249+0000", 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7449999999999998e-06 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792254+0000", 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4859999999999997e-06 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48342 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793113+0000", 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025230302, 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029245375000000001, 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48342 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793113+0000", 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793113+0000", 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793114+0000", 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3e-07 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793116+0000", 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3690000000000001e-06 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793124+0000", 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8199999999999997e-06 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793129+0000", 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4330000000000003e-06 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48407 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800077+0000", 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018266009999999999, 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02232901, 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48407 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800077+0000", 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800077+0000", 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800078+0000", 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7899999999999999e-07 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800080+0000", 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5000000000000002e-06 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800088+0000", 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6550000000000004e-06 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800094+0000", 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8490000000000002e-06 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48470 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801578+0000", 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.016765120000000001, 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02088021, 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48470 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801578+0000", 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801578+0000", 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801578+0000", 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1500000000000001e-07 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801580+0000", 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0529999999999999e-06 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801583+0000", 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1329999999999999e-06 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801585+0000", 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6240000000000002e-06 2024-09-12T07:20:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48534 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.808219+0000", 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010124361, 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014287845, 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48534 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808219+0000", 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808219+0000", 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808219+0000", 2024-09-12T07:20:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1600000000000002e-07 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808222+0000", 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1059999999999998e-06 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808228+0000", 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2169999999999999e-06 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808233+0000", 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.22e-06 2024-09-12T07:20:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48219 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791258+0000", 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027085127000000001, 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031302057000000001, 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48219 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791258+0000", 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791258+0000", 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791259+0000", 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1900000000000003e-07 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791261+0000", 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3700000000000002e-06 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791268+0000", 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1099999999999997e-06 2024-09-12T07:20:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791274+0000", 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4929999999999998e-06 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48284 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792261+0000", 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026082092000000001, 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030346563, 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48284 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792261+0000", 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792261+0000", 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792262+0000", 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9700000000000002e-07 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792264+0000", 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.278e-06 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792271+0000", 2024-09-12T07:20:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792276+0000", 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1100000000000002e-06 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48344 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793137+0000", 2024-09-12T07:20:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025206389999999999, 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029522944999999998, 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48344 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793137+0000", 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793137+0000", 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793137+0000", 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3799999999999998e-07 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793140+0000", 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793146+0000", 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6420000000000001e-06 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793152+0000", 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9020000000000004e-06 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48408 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800102+0000", 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018241611000000001, 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022606869000000002, 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48408 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800102+0000", 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800102+0000", 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800102+0000", 2024-09-12T07:20:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1899999999999998e-07 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800105+0000", 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3080000000000002e-06 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800113+0000", 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1790000000000008e-06 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800118+0000", 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5629999999999999e-06 2024-09-12T07:20:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48472 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801589+0000", 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.01675476, 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.021171063, 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48472 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801589+0000", 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801589+0000", 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807103+0000", 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055146350000000004 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807108+0000", 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7489999999999998e-06 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807121+0000", 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2548000000000001e-05 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807128+0000", 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5820000000000004e-06 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48536 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.808240+0000", 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010103592, 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014568895, 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48536 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808240+0000", 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.808240+0000", 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814029+0000", 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057895409999999996 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814034+0000", 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4360000000000004e-06 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814045+0000", 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0842999999999999e-05 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814052+0000", 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0770000000000002e-06 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48221 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791282+0000", 2024-09-12T07:20:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027061370000000001, 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031574107999999997, 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48221 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791282+0000", 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791282+0000", 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791283+0000", 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791285+0000", 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0820000000000001e-06 2024-09-12T07:20:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791292+0000", 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9489999999999999e-06 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791297+0000", 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3410000000000002e-06 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48285 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792284+0000", 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026059769, 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030630280999999999, 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48285 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792284+0000", 2024-09-12T07:20:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792284+0000", 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792284+0000", 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792286+0000", 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0540000000000002e-06 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792293+0000", 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9820000000000002e-06 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792298+0000", 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0590000000000002e-06 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48346 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793159+0000", 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025184094000000001, 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029803974, 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48346 2024-09-12T07:20:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793159+0000", 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793159+0000", 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793160+0000", 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1800000000000001e-07 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793162+0000", 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0229999999999999e-06 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793169+0000", 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793174+0000", 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4500000000000003e-06 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48410 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800251+0000", 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018091919000000001, 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022764217999999999, 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48410 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800251+0000", 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800251+0000", 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800252+0000", 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6199999999999997e-07 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800260+0000", 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6369999999999999e-06 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800271+0000", 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.112e-05 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800289+0000", 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8484000000000001e-05 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48474 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807141+0000", 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011201908, 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015921988000000002, 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48474 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807141+0000", 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807141+0000", 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807142+0000", 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200000000000002e-07 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807145+0000", 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7980000000000001e-06 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807152+0000", 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9110000000000004e-06 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807158+0000", 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7710000000000003e-06 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48538 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814065+0000", 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0042781690000000001, 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0090508180000000004, 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48538 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814065+0000", 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814065+0000", 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814066+0000", 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7300000000000001e-07 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814068+0000", 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2539999999999999e-06 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814074+0000", 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5459999999999997e-06 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814080+0000", 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.074e-06 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48223 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791304+0000", 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.027039001, 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031859929000000002, 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48223 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791304+0000", 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791304+0000", 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791305+0000", 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3599999999999999e-07 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791317+0000", 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2354999999999999e-05 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791331+0000", 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3722e-05 2024-09-12T07:20:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791337+0000", 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48287 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792314+0000", 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026029783000000001, 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030903743000000001, 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48287 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792314+0000", 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792314+0000", 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792314+0000", 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792317+0000", 2024-09-12T07:20:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3999999999999999e-06 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792324+0000", 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4830000000000002e-06 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792330+0000", 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48347 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793181+0000", 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025162442, 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030084412000000001, 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48347 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793181+0000", 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793181+0000", 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793181+0000", 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9999999999999998e-07 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793183+0000", 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8989999999999999e-06 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793190+0000", 2024-09-12T07:20:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0160000000000003e-06 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793196+0000", 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5709999999999997e-06 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48412 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800301+0000", 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.018042473999999999, 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023015858, 2024-09-12T07:20:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48412 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800301+0000", 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800301+0000", 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800301+0000", 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4600000000000005e-07 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800304+0000", 2024-09-12T07:20:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6879999999999999e-06 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800312+0000", 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9510000000000006e-06 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800319+0000", 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.567e-06 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48475 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807166+0000", 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011177464999999999, 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016198183000000001, 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48475 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807166+0000", 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807166+0000", 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807166+0000", 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:20:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807169+0000", 2024-09-12T07:20:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807175+0000", 2024-09-12T07:20:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6869999999999997e-06 2024-09-12T07:20:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807181+0000", 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8549999999999998e-06 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48540 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814087+0000", 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0042568600000000003, 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0093289619999999997, 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48540 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814087+0000", 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814087+0000", 2024-09-12T07:20:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814087+0000", 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6400000000000003e-07 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814089+0000", 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.978e-06 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814095+0000", 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0209999999999996e-06 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814100+0000", 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0649999999999998e-06 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48225 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791354+0000", 2024-09-12T07:20:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026989625999999999, 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032109496000000001, 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48225 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791354+0000", 2024-09-12T07:20:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791354+0000", 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791354+0000", 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9800000000000003e-07 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791357+0000", 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5390000000000002e-06 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791400+0000", 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3229999999999998e-05 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791414+0000", 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4307999999999999e-05 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48289 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792338+0000", 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026005474000000001, 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031177150000000001, 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48289 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792338+0000", 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792338+0000", 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792338+0000", 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7999999999999996e-07 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792340+0000", 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9400000000000001e-06 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792349+0000", 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9590000000000001e-06 2024-09-12T07:20:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792355+0000", 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4099999999999999e-06 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48349 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793203+0000", 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025140052, 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030359113, 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48349 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793203+0000", 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793203+0000", 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793204+0000", 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793206+0000", 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0540000000000002e-06 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793212+0000", 2024-09-12T07:20:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6930000000000001e-06 2024-09-12T07:20:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793219+0000", 2024-09-12T07:20:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0739999999999998e-06 2024-09-12T07:20:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48414 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800442+0000", 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017901088999999999, 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023174481, 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48414 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800442+0000", 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800442+0000", 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800443+0000", 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3499999999999996e-07 2024-09-12T07:20:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800446+0000", 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.174e-06 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800458+0000", 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1715999999999999e-05 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800476+0000", 2024-09-12T07:20:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7754e-05 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48478 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807188+0000", 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011155086, 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016477775, 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48478 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807188+0000", 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807188+0000", 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807189+0000", 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3799999999999996e-07 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807191+0000", 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0930000000000001e-06 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807198+0000", 2024-09-12T07:20:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5459999999999997e-06 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807203+0000", 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3410000000000002e-06 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48542 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814106+0000", 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0042373860000000001, 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096071420000000008, 2024-09-12T07:20:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48542 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814106+0000", 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814106+0000", 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814106+0000", 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4499999999999998e-07 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814108+0000", 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.773e-06 2024-09-12T07:20:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814114+0000", 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7459999999999997e-06 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814118+0000", 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4959999999999998e-06 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48227 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791425+0000", 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026917967000000001, 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032339286000000002, 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48227 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791425+0000", 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791425+0000", 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791426+0000", 2024-09-12T07:20:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.68e-07 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791429+0000", 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0489999999999999e-06 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791437+0000", 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1640000000000001e-06 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791447+0000", 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0107e-05 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48291 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792363+0000", 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025980824999999999, 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031450191000000002, 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48291 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792363+0000", 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792363+0000", 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792363+0000", 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7599999999999998e-07 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792365+0000", 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0549999999999998e-06 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792372+0000", 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7839999999999998e-06 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792377+0000", 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1440000000000002e-06 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.205 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48351 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793256+0000", 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025087795, 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030609411999999999, 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48351 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793256+0000", 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793256+0000", 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793256+0000", 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9400000000000005e-07 2024-09-12T07:20:00.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793259+0000", 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4370000000000001e-06 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793267+0000", 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2619999999999998e-06 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793274+0000", 2024-09-12T07:20:00.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5379999999999997e-06 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48416 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800488+0000", 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017855084, 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023432959, 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48416 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800488+0000", 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800488+0000", 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800489+0000", 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7199999999999999e-07 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800492+0000", 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7810000000000001e-06 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800514+0000", 2024-09-12T07:20:00.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2568000000000001e-05 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800523+0000", 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5220000000000007e-06 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48480 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807210+0000", 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011133245999999999, 2024-09-12T07:20:00.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016763169000000001, 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48480 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807210+0000", 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807210+0000", 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807211+0000", 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9500000000000003e-07 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807213+0000", 2024-09-12T07:20:00.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2419999999999999e-06 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807219+0000", 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1829999999999999e-06 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807224+0000", 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.215 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48544 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814124+0000", 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0042190309999999998, 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0098967110000000007, 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48544 2024-09-12T07:20:00.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814124+0000", 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814124+0000", 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814125+0000", 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814126+0000", 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.77e-06 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814132+0000", 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8130000000000001e-06 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814137+0000", 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4560000000000002e-06 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48229 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791456+0000", 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026887378, 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032617676999999998, 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.219 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48229 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791456+0000", 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791456+0000", 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791457+0000", 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9100000000000004e-07 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791487+0000", 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0645000000000002e-05 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791496+0000", 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7439999999999996e-06 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791502+0000", 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48293 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.222 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792384+0000", 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025959644, 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031737981999999998, 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48293 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792384+0000", 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792384+0000", 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792384+0000", 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7399999999999999e-07 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792386+0000", 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9199999999999998e-06 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792393+0000", 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4999999999999996e-06 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792398+0000", 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3040000000000004e-06 2024-09-12T07:20:00.225 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48353 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793285+0000", 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025058627999999999, 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.030891615000000001, 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.226 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48353 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793285+0000", 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793285+0000", 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793285+0000", 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3099999999999998e-07 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793288+0000", 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.305e-06 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793295+0000", 2024-09-12T07:20:00.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3390000000000004e-06 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793301+0000", 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3470000000000004e-06 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48418 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800532+0000", 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017811403999999999, 2024-09-12T07:20:00.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023692422000000001, 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48418 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800532+0000", 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800532+0000", 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800532+0000", 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7399999999999998e-07 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800535+0000", 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4849999999999999e-06 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.232 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.232 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800543+0000", 2024-09-12T07:20:00.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5320000000000006e-06 2024-09-12T07:20:00.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.232 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.232 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800550+0000", 2024-09-12T07:20:00.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0480000000000004e-06 2024-09-12T07:20:00.232 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.232 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.232 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48482 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807231+0000", 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011112535999999999, 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017047835000000001, 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48482 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.233 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807231+0000", 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807231+0000", 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807231+0000", 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8299999999999998e-07 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807239+0000", 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1210000000000007e-06 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807246+0000", 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7349999999999999e-06 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807251+0000", 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1000000000000003e-06 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48546 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814143+0000", 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0042007340000000002, 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010184201, 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.236 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48546 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814143+0000", 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814143+0000", 2024-09-12T07:20:00.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814143+0000", 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4299999999999999e-07 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814145+0000", 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8330000000000001e-06 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814150+0000", 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6470000000000003e-06 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814155+0000", 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3839999999999999e-06 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48232 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791533+0000", 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026810324999999999, 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032846118000000001, 2024-09-12T07:20:00.239 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48232 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791533+0000", 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791533+0000", 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791534+0000", 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1600000000000001e-07 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791536+0000", 2024-09-12T07:20:00.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791545+0000", 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2009999999999999e-06 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791551+0000", 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.003e-06 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48295 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792405+0000", 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025938648000000002, 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032022390999999997, 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48295 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.243 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792405+0000", 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792405+0000", 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792405+0000", 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2199999999999999e-07 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792412+0000", 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7159999999999997e-06 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792419+0000", 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6209999999999998e-06 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792424+0000", 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4500000000000003e-06 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48354 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793309+0000", 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025034554000000001, 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031170707999999998, 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.246 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48354 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793309+0000", 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793309+0000", 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793309+0000", 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5699999999999998e-07 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793311+0000", 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.029e-06 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793319+0000", 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2199999999999992e-06 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793325+0000", 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6110000000000001e-06 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48420 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.249 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800558+0000", 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017785492, 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023969396, 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48420 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800558+0000", 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800558+0000", 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800558+0000", 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9699999999999996e-07 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800561+0000", 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3609999999999999e-06 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800570+0000", 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0340000000000002e-06 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800601+0000", 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0830000000000001e-05 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.252 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48484 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807259+0000", 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011084549000000001, 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017319958, 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.253 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48484 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807259+0000", 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807259+0000", 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807259+0000", 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9100000000000004e-07 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807262+0000", 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4080000000000001e-06 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807268+0000", 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0079999999999999e-06 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807273+0000", 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2449999999999998e-06 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48547 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814161+0000", 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0041826509999999999, 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010467136, 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48547 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814161+0000", 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814161+0000", 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814161+0000", 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.41e-07 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814163+0000", 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7120000000000001e-06 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814169+0000", 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3570000000000003e-06 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814174+0000", 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5290000000000002e-06 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48233 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791596+0000", 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026746974, 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033079028000000003, 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48233 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791596+0000", 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791596+0000", 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791597+0000", 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791600+0000", 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0989999999999999e-06 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791611+0000", 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0703999999999999e-05 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791617+0000", 2024-09-12T07:20:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5969999999999997e-06 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48297 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792442+0000", 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025901495, 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032287914000000001, 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.263 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48297 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792442+0000", 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792442+0000", 2024-09-12T07:20:00.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792443+0000", 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.37e-07 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792445+0000", 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.278e-06 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792452+0000", 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6380000000000004e-06 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792458+0000", 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6200000000000004e-06 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48356 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.266 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793333+0000", 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025010787999999999, 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031447540000000003, 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48356 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793333+0000", 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793333+0000", 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793333+0000", 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9499999999999998e-07 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793335+0000", 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2259999999999999e-06 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793342+0000", 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2300000000000002e-06 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793348+0000", 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4530000000000001e-06 2024-09-12T07:20:00.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48422 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800620+0000", 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017722941999999998, 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024212183000000002, 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48422 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800620+0000", 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800620+0000", 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800621+0000", 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800624+0000", 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9490000000000001e-06 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800643+0000", 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8953999999999998e-05 2024-09-12T07:20:00.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800650+0000", 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5989999999999998e-06 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48486 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807280+0000", 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011063129, 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017605848, 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.273 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48486 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807280+0000", 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807280+0000", 2024-09-12T07:20:00.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807281+0000", 2024-09-12T07:20:00.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1899999999999998e-07 2024-09-12T07:20:00.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807283+0000", 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1409999999999999e-06 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807289+0000", 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5549999999999999e-06 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807294+0000", 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0119999999999996e-06 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48549 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814180+0000", 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0041637619999999997, 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01075866, 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48549 2024-09-12T07:20:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814180+0000", 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814180+0000", 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814180+0000", 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.53e-07 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814182+0000", 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.637e-06 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814187+0000", 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5899999999999998e-06 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814192+0000", 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4120000000000003e-06 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48235 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791627+0000", 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026716327000000002, 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033359853000000002, 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.280 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48235 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791627+0000", 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791627+0000", 2024-09-12T07:20:00.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791628+0000", 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7300000000000001e-07 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791630+0000", 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.052e-06 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791637+0000", 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1060000000000002e-06 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791642+0000", 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4e-06 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48298 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792467+0000", 2024-09-12T07:20:00.283 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025876667999999999, 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032572690000000001, 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48298 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792467+0000", 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792467+0000", 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792467+0000", 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9699999999999996e-07 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792470+0000", 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3240000000000001e-06 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792476+0000", 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9360000000000002e-06 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792482+0000", 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9209999999999997e-06 2024-09-12T07:20:00.286 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48358 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793357+0000", 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024986648, 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031733742000000002, 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.287 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48358 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793357+0000", 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793357+0000", 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793357+0000", 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5200000000000002e-07 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793359+0000", 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0499999999999999e-06 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793367+0000", 2024-09-12T07:20:00.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8699999999999992e-06 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793373+0000", 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9429999999999997e-06 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48424 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800668+0000", 2024-09-12T07:20:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017675682000000002, 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024503838999999999, 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48424 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800668+0000", 2024-09-12T07:20:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800668+0000", 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800668+0000", 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6899999999999997e-07 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800671+0000", 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6919999999999998e-06 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800690+0000", 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9476000000000001e-05 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800697+0000", 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1310000000000003e-06 2024-09-12T07:20:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48488 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807301+0000", 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011042411, 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017919617999999998, 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48488 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807301+0000", 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807301+0000", 2024-09-12T07:20:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807301+0000", 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.27e-07 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807303+0000", 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9130000000000001e-06 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807309+0000", 2024-09-12T07:20:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0800000000000002e-06 2024-09-12T07:20:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807314+0000", 2024-09-12T07:20:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1019999999999996e-06 2024-09-12T07:20:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48551 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814197+0000", 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0041461639999999999, 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011072343, 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48551 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814197+0000", 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814197+0000", 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814198+0000", 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.34e-07 2024-09-12T07:20:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814199+0000", 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6509999999999999e-06 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814205+0000", 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.694e-06 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814209+0000", 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5650000000000003e-06 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48237 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791650+0000", 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026693720000000001, 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033667750000000003, 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48237 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791650+0000", 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791650+0000", 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791650+0000", 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791652+0000", 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0329999999999998e-06 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791661+0000", 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2469999999999997e-06 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791667+0000", 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7440000000000004e-06 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48299 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792490+0000", 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025853028, 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032874368000000001, 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48299 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792490+0000", 2024-09-12T07:20:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792490+0000", 2024-09-12T07:20:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792491+0000", 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7100000000000002e-07 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792493+0000", 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0310000000000001e-06 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792500+0000", 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6950000000000003e-06 2024-09-12T07:20:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792505+0000", 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0429999999999998e-06 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48360 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793381+0000", 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024962577, 2024-09-12T07:20:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032032104999999998, 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48360 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793381+0000", 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793381+0000", 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793381+0000", 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0200000000000003e-07 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793390+0000", 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4999999999999999e-06 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793397+0000", 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1949999999999997e-06 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793403+0000", 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5710000000000003e-06 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48426 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800716+0000", 2024-09-12T07:20:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017627395000000001, 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024745629000000002, 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48426 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800716+0000", 2024-09-12T07:20:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800716+0000", 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800717+0000", 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2799999999999996e-07 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800719+0000", 2024-09-12T07:20:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7819999999999998e-06 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800727+0000", 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7780000000000007e-06 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800744+0000", 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6657999999999999e-05 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48490 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807321+0000", 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0110221, 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018188444000000002, 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48490 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807321+0000", 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807321+0000", 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807322+0000", 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:20:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807324+0000", 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0669999999999999e-06 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807330+0000", 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6250000000000001e-06 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807335+0000", 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8380000000000001e-06 2024-09-12T07:20:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48553 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814215+0000", 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0041282209999999996, 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011342744, 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48553 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814215+0000", 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814215+0000", 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814216+0000", 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.41e-07 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814222+0000", 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.888e-06 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814229+0000", 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0990000000000004e-06 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814233+0000", 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.668e-06 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48239 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791675+0000", 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026668547000000001, 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033931187000000002, 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48239 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791675+0000", 2024-09-12T07:20:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791675+0000", 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791675+0000", 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4200000000000001e-07 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791677+0000", 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0839999999999999e-06 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791685+0000", 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3590000000000002e-06 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791690+0000", 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2140000000000004e-06 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48301 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792512+0000", 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025831390999999999, 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033144163999999997, 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48301 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792512+0000", 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792512+0000", 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792512+0000", 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-07 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792514+0000", 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8279999999999999e-06 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792521+0000", 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8589999999999999e-06 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792557+0000", 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5497000000000003e-05 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48362 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793412+0000", 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024931228999999999, 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032292781, 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48362 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793412+0000", 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793412+0000", 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793413+0000", 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8299999999999997e-07 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793415+0000", 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1619999999999998e-06 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793421+0000", 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6159999999999998e-06 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793427+0000", 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4e-06 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48428 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800753+0000", 2024-09-12T07:20:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017590036999999999, 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024999549999999999, 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48428 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800753+0000", 2024-09-12T07:20:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800753+0000", 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800754+0000", 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800757+0000", 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.717e-06 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800779+0000", 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2855999999999999e-05 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800786+0000", 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5030000000000002e-06 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48492 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807343+0000", 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.011000874000000001, 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018457085000000002, 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48492 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807343+0000", 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807343+0000", 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807343+0000", 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807345+0000", 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0370000000000001e-06 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807351+0000", 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1029999999999998e-06 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807356+0000", 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48555 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814239+0000", 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0041041350000000001, 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0116087, 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48555 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814239+0000", 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814239+0000", 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814240+0000", 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1100000000000001e-07 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814241+0000", 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6530000000000001e-06 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814247+0000", 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7479999999999999e-06 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814252+0000", 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4920000000000004e-06 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48241 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791698+0000", 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026645288, 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.034212703999999997, 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48241 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791698+0000", 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791698+0000", 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791699+0000", 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:20:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791701+0000", 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2989999999999999e-06 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791708+0000", 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1330000000000001e-06 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791713+0000", 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2859999999999999e-06 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48303 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792576+0000", 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025767099000000002, 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033382606000000002, 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48303 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792576+0000", 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792576+0000", 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792577+0000", 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8999999999999996e-07 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792580+0000", 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.174e-06 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792588+0000", 2024-09-12T07:20:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2670000000000006e-06 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792594+0000", 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1630000000000001e-06 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48364 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793435+0000", 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024908705, 2024-09-12T07:20:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032571467999999999, 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48364 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793435+0000", 2024-09-12T07:20:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793435+0000", 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793435+0000", 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793437+0000", 2024-09-12T07:20:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1100000000000001e-06 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793448+0000", 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0485e-05 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793454+0000", 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8080000000000001e-06 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48430 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800805+0000", 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017538237000000002, 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025249588, 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48430 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800805+0000", 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800805+0000", 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800806+0000", 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6000000000000004e-07 2024-09-12T07:20:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800809+0000", 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.92e-06 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800817+0000", 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3869999999999995e-06 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800823+0000", 2024-09-12T07:20:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0190000000000003e-06 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48494 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807569+0000", 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010774275999999999, 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018534278000000001, 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48494 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807569+0000", 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807569+0000", 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807570+0000", 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1099999999999995e-07 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807573+0000", 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7620000000000001e-06 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807584+0000", 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0869e-05 2024-09-12T07:20:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807591+0000", 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.72e-06 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48557 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814494+0000", 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0038488950000000002, 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011656764, 2024-09-12T07:20:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48557 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814494+0000", 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814494+0000", 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814495+0000", 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3600000000000004e-07 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814498+0000", 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0350000000000002e-06 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814506+0000", 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8359999999999992e-06 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814511+0000", 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4500000000000003e-06 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48243 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791720+0000", 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026622979000000001, 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.034478389999999998, 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48243 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791720+0000", 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791720+0000", 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791721+0000", 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4499999999999997e-07 2024-09-12T07:20:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791723+0000", 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9520000000000001e-06 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791729+0000", 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4439999999999996e-06 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791734+0000", 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.113e-06 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48305 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792604+0000", 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025739563, 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033643595999999998, 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48305 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792604+0000", 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792604+0000", 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792604+0000", 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7300000000000001e-07 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792607+0000", 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1969999999999999e-06 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792614+0000", 2024-09-12T07:20:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1219999999999997e-06 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792619+0000", 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0819999999999998e-06 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48367 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793462+0000", 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024881555999999999, 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.032833955999999997, 2024-09-12T07:20:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48367 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793462+0000", 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793462+0000", 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793462+0000", 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0100000000000005e-07 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793465+0000", 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2529999999999999e-06 2024-09-12T07:20:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793473+0000", 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.32e-06 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793478+0000", 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4739999999999996e-06 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48432 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800881+0000", 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017462061000000001, 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025462293, 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48432 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800881+0000", 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800881+0000", 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800882+0000", 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1500000000000004e-07 2024-09-12T07:20:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800885+0000", 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0790000000000001e-06 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800895+0000", 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8609999999999993e-06 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800902+0000", 2024-09-12T07:20:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8159999999999996e-06 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48496 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807602+0000", 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010741313000000001, 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018791022000000001, 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48496 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807602+0000", 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807602+0000", 2024-09-12T07:20:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807603+0000", 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5899999999999996e-07 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807606+0000", 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9000000000000002e-06 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807613+0000", 2024-09-12T07:20:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9789999999999996e-06 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807618+0000", 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7570000000000001e-06 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48559 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814519+0000", 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0038242509999999999, 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011921877000000001, 2024-09-12T07:20:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48559 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814519+0000", 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814519+0000", 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814520+0000", 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5499999999999998e-07 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814521+0000", 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8789999999999999e-06 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814527+0000", 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6069999999999998e-06 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814532+0000", 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4660000000000001e-06 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48245 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791742+0000", 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026601886000000002, 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.034747781999999998, 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48245 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791742+0000", 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791742+0000", 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791742+0000", 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791744+0000", 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.108e-06 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791761+0000", 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7404000000000001e-05 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791769+0000", 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0199999999999997e-06 2024-09-12T07:20:00.385 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48307 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792626+0000", 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025717394000000001, 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033910816000000003, 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.386 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48307 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792626+0000", 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792626+0000", 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792626+0000", 2024-09-12T07:20:00.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4200000000000001e-07 2024-09-12T07:20:00.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792628+0000", 2024-09-12T07:20:00.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9259999999999999e-06 2024-09-12T07:20:00.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792635+0000", 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0110000000000003e-06 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792641+0000", 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4e-06 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48369 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.389 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793486+0000", 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024857462, 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033098285999999998, 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48369 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793486+0000", 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793486+0000", 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793486+0000", 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9200000000000002e-07 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793489+0000", 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.199e-06 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793497+0000", 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2339999999999994e-06 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793502+0000", 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5509999999999999e-06 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.392 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48434 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800921+0000", 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017422732, 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025711595, 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.393 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48434 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800921+0000", 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800921+0000", 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800921+0000", 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9200000000000001e-07 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800929+0000", 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4499999999999998e-06 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800937+0000", 2024-09-12T07:20:00.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9130000000000003e-06 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800943+0000", 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.054e-06 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48498 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807626+0000", 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010717244000000001, 2024-09-12T07:20:00.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019040423000000001, 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48498 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807626+0000", 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807626+0000", 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807627+0000", 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1600000000000002e-07 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807629+0000", 2024-09-12T07:20:00.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2330000000000001e-06 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807635+0000", 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8769999999999998e-06 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807640+0000", 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4439999999999999e-06 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48561 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.814537+0000", 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0038060479999999998, 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012153809999999999, 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48561 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.400 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814537+0000", 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814537+0000", 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814538+0000", 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2800000000000003e-07 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814540+0000", 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8309999999999999e-06 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814545+0000", 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.541e-06 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.814549+0000", 2024-09-12T07:20:00.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.25e-06 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48247 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791820+0000", 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026523105000000002, 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.034895079000000002, 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.403 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48247 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791820+0000", 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791820+0000", 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791821+0000", 2024-09-12T07:20:00.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4700000000000001e-07 2024-09-12T07:20:00.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791828+0000", 2024-09-12T07:20:00.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6580000000000002e-06 2024-09-12T07:20:00.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791838+0000", 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2539999999999998e-06 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791844+0000", 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.567e-06 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48309 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.406 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792648+0000", 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025695592999999999, 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.034091532000000001, 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48309 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792648+0000", 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792648+0000", 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792648+0000", 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9200000000000002e-07 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792650+0000", 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.885e-06 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792657+0000", 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8140000000000003e-06 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792662+0000", 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2100000000000001e-06 2024-09-12T07:20:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48371 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793526+0000", 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024817470000000001, 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033237416999999998, 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.410 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48371 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793526+0000", 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793526+0000", 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793526+0000", 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.58e-07 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793529+0000", 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4109999999999999e-06 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793536+0000", 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6040000000000004e-06 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793542+0000", 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0299999999999999e-06 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48436 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800962+0000", 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017381420000000002, 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025824988, 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.413 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48436 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800962+0000", 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800962+0000", 2024-09-12T07:20:00.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800962+0000", 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2e-07 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800965+0000", 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.497e-06 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800972+0000", 2024-09-12T07:20:00.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4460000000000004e-06 2024-09-12T07:20:00.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800978+0000", 2024-09-12T07:20:00.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.84e-06 2024-09-12T07:20:00.416 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.416 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.416 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48501 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807647+0000", 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010695945, 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019163547, 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48501 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807647+0000", 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807647+0000", 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807648+0000", 2024-09-12T07:20:00.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6100000000000002e-07 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807650+0000", 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.137e-06 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807656+0000", 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3929999999999996e-06 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807662+0000", 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.344e-06 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48249 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791853+0000", 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026489928999999999, 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.034983698000000001, 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.420 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48249 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791853+0000", 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791853+0000", 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791854+0000", 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6100000000000001e-07 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791856+0000", 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3520000000000001e-06 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791866+0000", 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3009999999999995e-06 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791879+0000", 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3295000000000001e-05 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48311 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.423 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792684+0000", 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025659234999999999, 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.034176785000000001, 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48311 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.424 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792684+0000", 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792684+0000", 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792685+0000", 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6000000000000004e-07 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792688+0000", 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8219999999999998e-06 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792698+0000", 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0623e-05 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792704+0000", 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8459999999999996e-06 2024-09-12T07:20:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48373 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793551+0000", 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024792649, 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033356098000000001, 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.427 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48373 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793551+0000", 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793551+0000", 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793551+0000", 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793553+0000", 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9250000000000002e-06 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793560+0000", 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9759999999999998e-06 2024-09-12T07:20:00.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793565+0000", 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3249999999999998e-06 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48437 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.800986+0000", 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017357320999999998, 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025958800000000001, 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48437 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800986+0000", 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800986+0000", 2024-09-12T07:20:00.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800987+0000", 2024-09-12T07:20:00.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5900000000000002e-07 2024-09-12T07:20:00.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.800994+0000", 2024-09-12T07:20:00.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5900000000000002e-06 2024-09-12T07:20:00.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801002+0000", 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6420000000000007e-06 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801008+0000", 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8050000000000003e-06 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48504 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807669+0000", 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010674256999999999, 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019306944999999999, 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48504 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.434 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807669+0000", 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807669+0000", 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807669+0000", 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3599999999999999e-07 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807671+0000", 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9700000000000002e-06 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807678+0000", 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6830000000000002e-06 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807683+0000", 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2429999999999996e-06 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48250 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791888+0000", 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026454894999999999, 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.035128673999999999, 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.437 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48250 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791888+0000", 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791888+0000", 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791889+0000", 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.06e-07 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791892+0000", 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.74e-06 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791899+0000", 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6599999999999995e-06 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791905+0000", 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8139999999999997e-06 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.440 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48313 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792718+0000", 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025625815999999999, 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.034345674999999999, 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48313 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792718+0000", 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792718+0000", 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792718+0000", 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8299999999999997e-07 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792720+0000", 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.34e-06 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792729+0000", 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9970000000000004e-06 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792735+0000", 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9689999999999999e-06 2024-09-12T07:20:00.443 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48375 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793573+0000", 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024770374000000001, 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033538125000000002, 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.444 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48375 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793573+0000", 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793573+0000", 2024-09-12T07:20:00.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793573+0000", 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7500000000000001e-07 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793576+0000", 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2069999999999998e-06 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793582+0000", 2024-09-12T07:20:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8739999999999997e-06 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793608+0000", 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5613999999999999e-05 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48439 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801016+0000", 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017327879000000001, 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026143921000000001, 2024-09-12T07:20:00.447 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48439 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801016+0000", 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801016+0000", 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801016+0000", 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8400000000000005e-07 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801018+0000", 2024-09-12T07:20:00.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9350000000000001e-06 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801025+0000", 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0280000000000003e-06 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801030+0000", 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4090000000000002e-06 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48506 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807691+0000", 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010652880999999999, 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019515412999999999, 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48506 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.451 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807691+0000", 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807691+0000", 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807691+0000", 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4499999999999998e-07 2024-09-12T07:20:00.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807693+0000", 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3350000000000001e-06 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807699+0000", 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2040000000000002e-06 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807705+0000", 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1549999999999998e-06 2024-09-12T07:20:00.453 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48253 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791913+0000", 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.026430801, 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.035340073999999999, 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.454 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48253 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791913+0000", 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791913+0000", 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791913+0000", 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.32e-07 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791915+0000", 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0940000000000002e-06 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791922+0000", 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8430000000000004e-06 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791927+0000", 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1529999999999996e-06 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48314 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792743+0000", 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025599942000000001, 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.034537136000000003, 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.457 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48314 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792743+0000", 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792743+0000", 2024-09-12T07:20:00.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792744+0000", 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9699999999999996e-07 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792746+0000", 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1519999999999999e-06 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792755+0000", 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7309999999999999e-06 2024-09-12T07:20:00.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792760+0000", 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6359999999999999e-06 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48377 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793618+0000", 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024725595, 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033688343000000003, 2024-09-12T07:20:00.460 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48377 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793618+0000", 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793618+0000", 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793618+0000", 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3500000000000002e-07 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793621+0000", 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8059999999999999e-06 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793628+0000", 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0999999999999998e-06 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793634+0000", 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.434e-06 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48441 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.463 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801038+0000", 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017305838, 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026294455000000001, 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48441 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801038+0000", 2024-09-12T07:20:00.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801038+0000", 2024-09-12T07:20:00.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801038+0000", 2024-09-12T07:20:00.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5699999999999998e-07 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801040+0000", 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8780000000000001e-06 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801047+0000", 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0500000000000003e-06 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801052+0000", 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5099999999999998e-06 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48508 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807711+0000", 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010632206, 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019650833999999999, 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.467 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48508 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807711+0000", 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807711+0000", 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807712+0000", 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807718+0000", 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8730000000000001e-06 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807725+0000", 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9160000000000004e-06 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807731+0000", 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4240000000000001e-06 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.470 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48256 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.791934+0000", 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.02640899, 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.035472050999999998, 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48256 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791934+0000", 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791934+0000", 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791935+0000", 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1199999999999998e-07 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.472 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791937+0000", 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.181e-06 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791944+0000", 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9269999999999999e-06 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.791949+0000", 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4430000000000002e-06 2024-09-12T07:20:00.473 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48316 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.792775+0000", 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.025568744000000001, 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.034680200000000001, 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.474 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48316 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792775+0000", 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792775+0000", 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792775+0000", 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9699999999999996e-07 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792783+0000", 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7810000000000005e-06 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792791+0000", 2024-09-12T07:20:00.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7379999999999994e-06 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.792797+0000", 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.72e-06 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48379 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.793643+0000", 2024-09-12T07:20:00.477 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.024700241000000001, 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033859577000000002, 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48379 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793643+0000", 2024-09-12T07:20:00.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793643+0000", 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793644+0000", 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6199999999999998e-07 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793646+0000", 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2180000000000002e-06 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793653+0000", 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6279999999999996e-06 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.793660+0000", 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9840000000000004e-06 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.480 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48443 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.801111+0000", 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.017232641, 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026441503000000002, 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.481 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48443 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801111+0000", 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801111+0000", 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801111+0000", 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9400000000000005e-07 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801114+0000", 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.055e-06 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801126+0000", 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1701e-05 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.801133+0000", 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7100000000000001e-06 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:48510 3.13 3.1c445293 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.807844+0000", 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.010499316999999999, 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019755865000000001, 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:00.484 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 48510 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807844+0000", 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807844+0000", 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807845+0000", 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8999999999999996e-07 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807848+0000", 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6830000000000001e-06 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807859+0000", 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0971999999999999e-05 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.807873+0000", 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3570999999999999e-05 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 194 2024-09-12T07:20:00.487 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:00.488 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:00.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:00.490 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:00.490+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.2 to 1 (10000) 2024-09-12T07:20:00.491 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-12T07:20:00.589 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:00.590+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:00.673 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:00.673 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:00.682 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:00.723 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:00.725+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:00.737 INFO:teuthology.orchestra.run.smithi046.stdout:912 writesame 0x5939368 thru 0x5948301 data_size 0x3(0xef9a bytes) 2024-09-12T07:20:00.756 INFO:teuthology.orchestra.run.smithi046.stdout:915 write 0x55d1655 thru 0x55df5ae (0xdf5a bytes) 2024-09-12T07:20:00.760 INFO:teuthology.orchestra.run.smithi046.stdout:916 punch from 0x1405537 to 0x14091a7, (0x3c70 bytes) 2024-09-12T07:20:00.760 INFO:teuthology.orchestra.run.smithi046.stdout:917 write 0x664759b thru 0x6647b35 (0x59b bytes) 2024-09-12T07:20:00.763 INFO:teuthology.orchestra.run.smithi046.stdout:918 read 0x5d90ec0 thru 0x5d9c818 (0xb959 bytes) 2024-09-12T07:20:00.763 INFO:teuthology.orchestra.run.smithi046.stdout:920 read 0x9bce9e1 thru 0x9bd3a35 (0x5055 bytes) 2024-09-12T07:20:00.763 INFO:teuthology.orchestra.run.smithi046.stdout:921 write 0x8ad0d91 thru 0x8adf118 (0xe388 bytes) 2024-09-12T07:20:00.767 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:00.767 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:00.767 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:00.767 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:00.767 INFO:teuthology.orchestra.run.smithi046.stdout:927 read 0x6afebfc thru 0x6b0e907 (0xfd0c bytes) 2024-09-12T07:20:00.767 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:00.767 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:00.767 INFO:teuthology.orchestra.run.smithi046.stdout:933 read 0x96a5eac thru 0x96adf97 (0x80ec bytes) 2024-09-12T07:20:00.768 INFO:teuthology.orchestra.run.smithi046.stdout:934 write 0x463d016 thru 0x4648e41 (0xbe2c bytes) 2024-09-12T07:20:00.780 INFO:teuthology.orchestra.run.smithi046.stdout:935 read 0xb0bbe8b thru 0xb0c83d7 (0xc54d bytes) 2024-09-12T07:20:00.780 INFO:teuthology.orchestra.run.smithi046.stdout:940 write 0x192c578 thru 0x193a174 (0xdbfd bytes) 2024-09-12T07:20:00.790 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:00.791+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:00.801 INFO:teuthology.orchestra.run.smithi046.stdout:941 punch from 0x56f617f to 0x56fa9ff, (0x4880 bytes) 2024-09-12T07:20:00.802 INFO:teuthology.orchestra.run.smithi046.stdout:942 clone 7 order 20 su 16384 sc 14 2024-09-12T07:20:00.836 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:00.836 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:00.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:00.891 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:00.893+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:00.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:00.992 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:00.994+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:01.069 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:01.069 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:01.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:01.093 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:01.095+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:01.140 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.4 to 1 (10000) 2024-09-12T07:20:01.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:20:01.155 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-12T07:20:01.192 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:01.193 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:01.195 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:01.196+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:01.203 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:01.246 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone6 from 0xc0f3d2c (overlap 0xc0f3d2c) to 0x9a5ca89 2024-09-12T07:20:01.295 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:01.296+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:01.368 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:01.368 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:01.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:01.419 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:01.420+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:01.463 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-12T07:20:01.464 INFO:teuthology.orchestra.run.smithi046.stdout:943 writesame 0x5574ba thru 0x55958d data_size 0x16(0x20d4 bytes) 2024-09-12T07:20:01.492 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:01.494 INFO:teuthology.orchestra.run.smithi046.stdout:947 trunc from 0xc0f3d2c to 0xa031006 2024-09-12T07:20:01.496 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:01.498+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:01.536 INFO:teuthology.orchestra.run.smithi046.stdout:948 read 0x6371f55 thru 0x6381822 (0xf8ce bytes) 2024-09-12T07:20:01.537 INFO:teuthology.orchestra.run.smithi046.stdout:949 writesame 0x92d6ca2 thru 0x92d7dfa data_size 0x1(0x1159 bytes) 2024-09-12T07:20:01.544 INFO:teuthology.orchestra.run.smithi046.stdout:950 write 0x5a8a2b4 thru 0x5a98e64 (0xebb1 bytes) 2024-09-12T07:20:01.548 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:01.548 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:01.562 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:01.566 INFO:teuthology.orchestra.run.smithi046.stdout:951 read 0x1af0e00 thru 0x1afc7fc (0xb9fd bytes) 2024-09-12T07:20:01.567 INFO:teuthology.orchestra.run.smithi046.stdout:953 writesame 0x618a9ab thru 0x619517e data_size 0x4(0xa7d4 bytes) 2024-09-12T07:20:01.576 INFO:teuthology.orchestra.run.smithi046.stdout:954 trunc from 0xa031006 to 0xc8691da 2024-09-12T07:20:01.584 INFO:teuthology.orchestra.run.smithi046.stdout:956 trunc from 0xc8691da to 0xa5bc74d 2024-09-12T07:20:01.594 INFO:teuthology.orchestra.run.smithi046.stdout:958 read 0x3e475b6 thru 0x3e4f40a (0x7e55 bytes) 2024-09-12T07:20:01.595 INFO:teuthology.orchestra.run.smithi046.stdout:959 write 0x189c76f thru 0x189e482 (0x1d14 bytes) 2024-09-12T07:20:01.597 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:01.598+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:01.605 INFO:teuthology.orchestra.run.smithi046.stdout:960 read 0x856ce70 thru 0x85707d3 (0x3964 bytes) 2024-09-12T07:20:01.605 INFO:teuthology.orchestra.run.smithi046.stdout:961 clone 8 order 23 su 32768 sc 15 2024-09-12T07:20:01.676 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:01.676 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:01.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:01.726 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:01.727+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:01.799 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:01.801+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:01.832 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:01.832 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:01.841 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:01.900 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:01.902+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:01.987 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:01.988 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:01.996 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:02.001 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:02.003+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:02.102 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:02.103+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:02.104 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:02.104 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:02.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:02.144 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:20:02.144 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.5 to 1 (10000) 2024-09-12T07:20:02.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-12T07:20:02.203 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:02.204+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:02.235 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:02.235 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:02.244 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:02.265 INFO:teuthology.orchestra.run.smithi046.stdout:leaving image image_client.0-clone7 intact 2024-09-12T07:20:02.309 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:02.310+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:02.381 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:02.381 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:02.390 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:02.405 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:02.407+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:02.481 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-12T07:20:02.492 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:02.506 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:02.507+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:02.574 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:02.574 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:02.584 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:02.628 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:02.630+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:02.632 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:02.633 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:02.650 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:02.666 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:02.667 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:02.683 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:02.688 INFO:teuthology.orchestra.run.smithi046.stdout:963 write 0x5de6a83 thru 0x5df2b4b (0xc0c9 bytes) 2024-09-12T07:20:02.700 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:02.701 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:02.707 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:02.708+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:02.708 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:02.708 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:02.714 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:02.740 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:02.768 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:02.768 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:02.785 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:02.785 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:02.807 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:02.809+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:02.814 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:02.814 INFO:teuthology.orchestra.run.smithi046.stdout:966 punch from 0x33556a4 to 0x335a146, (0x4aa2 bytes) 2024-09-12T07:20:02.814 INFO:teuthology.orchestra.run.smithi046.stdout:967 read 0x11f3e65 thru 0x11fa197 (0x6333 bytes) 2024-09-12T07:20:02.815 INFO:teuthology.orchestra.run.smithi046.stdout:968 read 0x5bf2b16 thru 0x5c02960 (0xfe4b bytes) 2024-09-12T07:20:02.815 INFO:teuthology.orchestra.run.smithi046.stdout:969 clone 9 order 20 su 65536 sc 10 2024-09-12T07:20:02.866 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:02.875 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:02.915 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:02.917+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:03.009 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:03.011+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:03.022 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:592 3.f e28/24 3:f4a2090e:::rbd_object_map.116011b0bca6:head v 28'17, mlcod=28'14)", 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.678290+0000", 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 15.345426947, 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0067778999999999999, 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678290+0000", 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678290+0000", 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678291+0000", 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.68e-07 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678313+0000", 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1888000000000001e-05 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678348+0000", 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5063e-05 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678373+0000", 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4973e-05 2024-09-12T07:20:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678448+0000", 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5743999999999999e-05 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678482+0000", 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3318999999999997e-05 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685027+0000", 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0065449169999999999 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685068+0000", 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1328000000000001e-05 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:594 3.f e28/24 3:f4a2090e:::rbd_object_map.116011b0bca6:head v 28'18, mlcod=28'14)", 2024-09-12T07:20:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.678916+0000", 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 15.3448008, 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006184691, 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678916+0000", 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678916+0000", 2024-09-12T07:20:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678917+0000", 2024-09-12T07:20:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8500000000000002e-07 2024-09-12T07:20:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678921+0000", 2024-09-12T07:20:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9330000000000003e-06 2024-09-12T07:20:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678929+0000", 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9980000000000003e-06 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678933+0000", 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4229999999999998e-06 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.678984+0000", 2024-09-12T07:20:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0831000000000002e-05 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679007+0000", 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3430000000000001e-05 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685070+0000", 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00606238 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685101+0000", 2024-09-12T07:20:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1211000000000003e-05 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:600 3.f e28/24 3:f4a2090e:::rbd_object_map.116011b0bca6:head v 28'19, mlcod=28'14)", 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:47.679546+0000", 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 15.344170833, 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059521690000000002, 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679546+0000", 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679546+0000", 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679547+0000", 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9200000000000001e-07 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679552+0000", 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6239999999999999e-06 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679564+0000", 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1704e-05 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679572+0000", 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7170000000000008e-06 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679626+0000", 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3841000000000002e-05 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.679639+0000", 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3806e-05 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685471+0000", 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058312909999999997 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:47.685498+0000", 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7594e-05 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1332 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'21, mlcod=29'17)", 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.747073+0000", 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 14.276644072, 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0076436810000000003, 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747073+0000", 2024-09-12T07:20:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747073+0000", 2024-09-12T07:20:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747073+0000", 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4899999999999995e-07 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747077+0000", 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6270000000000002e-06 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747086+0000", 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.302e-06 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747091+0000", 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4839999999999998e-06 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747137+0000", 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5896000000000002e-05 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747157+0000", 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-05 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754641+0000", 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0074841810000000003 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754717+0000", 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5562000000000006e-05 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1333 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'22, mlcod=29'17)", 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.747710+0000", 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 14.276007347, 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070469909999999998, 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747710+0000", 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747710+0000", 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747710+0000", 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747713+0000", 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1300000000000001e-06 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747720+0000", 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7179999999999999e-06 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747723+0000", 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5190000000000001e-06 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747773+0000", 2024-09-12T07:20:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9549000000000003e-05 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.747795+0000", 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2450999999999999e-05 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754722+0000", 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0069261949999999996 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754757+0000", 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4995e-05 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1334 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'23, mlcod=29'18)", 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.748316+0000", 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 14.275401136999999, 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0064707979999999998, 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748316+0000", 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748316+0000", 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748316+0000", 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748320+0000", 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7699999999999999e-06 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748329+0000", 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4100000000000008e-06 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748333+0000", 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2729999999999996e-06 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748376+0000", 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3464000000000003e-05 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748398+0000", 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1348999999999998e-05 2024-09-12T07:20:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754758+0000", 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0063606840000000001 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754787+0000", 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8357999999999999e-05 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1335 3.1c e29/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 29'24, mlcod=29'18)", 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:48.748830+0000", 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 14.274886529, 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059871769999999998, 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748830+0000", 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748830+0000", 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748831+0000", 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7999999999999995e-07 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748835+0000", 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1139999999999999e-06 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748845+0000", 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4569999999999994e-06 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748849+0000", 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6009999999999997e-06 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748901+0000", 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2280000000000001e-05 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.748922+0000", 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0047999999999998e-05 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754788+0000", 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058663999999999999 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:48.754818+0000", 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9697e-05 2024-09-12T07:20:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1687 3.1c e30/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 30'71, mlcod=30'68)", 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:49.557056+0000", 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.466661394999999, 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.006603757, 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557056+0000", 2024-09-12T07:20:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557056+0000", 2024-09-12T07:20:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557056+0000", 2024-09-12T07:20:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0200000000000002e-07 2024-09-12T07:20:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557062+0000", 2024-09-12T07:20:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9850000000000003e-06 2024-09-12T07:20:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557071+0000", 2024-09-12T07:20:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1770000000000004e-06 2024-09-12T07:20:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557076+0000", 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4689999999999999e-06 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557117+0000", 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1189999999999997e-05 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557134+0000", 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6637999999999999e-05 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563625+0000", 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0064912679999999997 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563659+0000", 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4428000000000003e-05 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:1688 3.1c e30/24 3:3b53d0bd:::rbd_object_map.116075042ea1:head v 30'72, mlcod=30'68)", 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:49.557723+0000", 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.465993922999999, 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059720429999999998, 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557723+0000", 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557723+0000", 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557724+0000", 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7599999999999996e-07 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557730+0000", 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1290000000000001e-06 2024-09-12T07:20:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557744+0000", 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3862999999999999e-05 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557751+0000", 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4070000000000004e-06 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557806+0000", 2024-09-12T07:20:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4379000000000002e-05 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.557827+0000", 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1410999999999999e-05 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563662+0000", 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0058354230000000002 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.563695+0000", 2024-09-12T07:20:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2654999999999998e-05 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:2067 3.1d e30/24 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:head v 30'14, mlcod=28'12)", 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:49.858445+0000", 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.165271520999999, 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059962629999999999, 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.858445+0000", 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.858445+0000", 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.858446+0000", 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3600000000000003e-07 2024-09-12T07:20:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859175+0000", 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00072900199999999997 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859191+0000", 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5376e-05 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859198+0000", 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4989999999999997e-06 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859263+0000", 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4519999999999999e-05 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.859281+0000", 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7971000000000001e-05 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.864380+0000", 2024-09-12T07:20:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050996720000000004 2024-09-12T07:20:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:49.864442+0000", 2024-09-12T07:20:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1487000000000003e-05 2024-09-12T07:20:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.4104062170000002, 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:20:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40356 3.11 e37/24 3:894561fa:::rbd_data.116063d335b3.0000000000000004:head v 37'30, mlcod=37'28)", 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:58.014063+0000", 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0096544679999999, 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059075530000000003, 2024-09-12T07:20:03.072 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014063+0000", 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014063+0000", 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014063+0000", 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5000000000000002e-07 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014072+0000", 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2989999999999997e-06 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014082+0000", 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0000000000000001e-05 2024-09-12T07:20:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014088+0000", 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3369999999999996e-06 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014138+0000", 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0111000000000002e-05 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014168+0000", 2024-09-12T07:20:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9802e-05 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.019936+0000", 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057680209999999999 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.019970+0000", 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4233000000000002e-05 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40359 3.15 e37/33 3:ac8f5600:::rbd_data.116063d335b3.0000000000000007:head v 37'12, mlcod=37'10)", 2024-09-12T07:20:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:58.014197+0000", 2024-09-12T07:20:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0095195060000002, 2024-09-12T07:20:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057866949999999997, 2024-09-12T07:20:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014197+0000", 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014197+0000", 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014198+0000", 2024-09-12T07:20:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.06e-07 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014203+0000", 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2959999999999998e-06 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014212+0000", 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0489999999999993e-06 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014219+0000", 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4400000000000002e-06 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014282+0000", 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3088000000000003e-05 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.014301+0000", 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8950999999999999e-05 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.019943+0000", 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056423050000000002 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:58.019984+0000", 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000001e-05 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40677 3.11 e39/24 3:8ca86d44:::rbd_data.116063d335b3.0000000000000003:head v 39'35, mlcod=37'32)", 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.232207+0000", 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 3.791510025, 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.007262147, 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232207+0000", 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232207+0000", 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232209+0000", 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.621e-06 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232217+0000", 2024-09-12T07:20:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7490000000000004e-06 2024-09-12T07:20:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232234+0000", 2024-09-12T07:20:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6528000000000002e-05 2024-09-12T07:20:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232243+0000", 2024-09-12T07:20:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2439999999999999e-06 2024-09-12T07:20:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232312+0000", 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8651000000000004e-05 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232335+0000", 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2903000000000001e-05 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.239414+0000", 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0070788919999999998 2024-09-12T07:20:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.239469+0000", 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5559000000000001e-05 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40674 3.13 e39/24 3:c93134b0:::rbd_data.116063d335b3.0000000000000000:head v 39'60, mlcod=38'57)", 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.232328+0000", 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 3.7913885270000001, 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0089405749999999992, 2024-09-12T07:20:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232328+0000", 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232328+0000", 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232329+0000", 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4299999999999999e-07 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232337+0000", 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0409999999999998e-06 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232346+0000", 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6850000000000007e-06 2024-09-12T07:20:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232351+0000", 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7030000000000003e-06 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232424+0000", 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2286000000000006e-05 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232442+0000", 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8354000000000001e-05 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.241227+0000", 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0087850240000000007 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.241269+0000", 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2138999999999998e-05 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40676 3.14 e39/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 39'18, mlcod=38'15)", 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.232370+0000", 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 3.7913469640000002, 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0085171880000000002, 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232370+0000", 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232370+0000", 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232370+0000", 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.3999999999999997e-07 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232378+0000", 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8339999999999999e-06 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232385+0000", 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8650000000000003e-06 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232390+0000", 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0089999999999998e-06 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232444+0000", 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4057000000000001e-05 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.232464+0000", 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0041999999999999e-05 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.240850+0000", 2024-09-12T07:20:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083861769999999999 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.240887+0000", 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6863999999999998e-05 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.27494596100000002, 2024-09-12T07:20:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:20:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.27071884000000002, 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:20:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:20:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "pg_update_log_missing(3.15 epoch 42/33 rep_tid 503 entries 42'27 (0'0) error 3:a9f25dc3:::rbd_header.1160f022dd61:head by client.4448.0:63197 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 42'25)", 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.765562+0000", 2024-09-12T07:20:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.25815522000000002, 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0090037820000000001, 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765562+0000", 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765562+0000", 2024-09-12T07:20:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765563+0000", 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.116e-06 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765577+0000", 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3736e-05 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765609+0000", 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2078e-05 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765627+0000", 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8181000000000001e-05 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765738+0000", 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111178 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.774566+0000", 2024-09-12T07:20:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0088274930000000005 2024-09-12T07:20:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63198 3.f e42/24 3:f45b9044:::rbd_object_map.1160f022dd61:head v 42'168, mlcod=42'166)", 2024-09-12T07:20:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.765781+0000", 2024-09-12T07:20:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.25793587699999998, 2024-09-12T07:20:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0092797550000000006, 2024-09-12T07:20:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765781+0000", 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765781+0000", 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765782+0000", 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7999999999999995e-07 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765787+0000", 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4940000000000003e-06 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765794+0000", 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9689999999999997e-06 2024-09-12T07:20:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765823+0000", 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8345999999999999e-05 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765892+0000", 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9052000000000002e-05 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.765915+0000", 2024-09-12T07:20:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3051999999999999e-05 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.775007+0000", 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0090927379999999995 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.775061+0000", 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3523999999999998e-05 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.113 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:03.114 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:03.114 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:03.116+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:03.145 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.6 to 1 (10000) 2024-09-12T07:20:03.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-12T07:20:03.214 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:03.216+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:03.260 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone8 from 0xa5bc74d (overlap 0xa5bc74d) to 0x0 2024-09-12T07:20:03.273 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63622 3.5 3.fdcf8ac5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270566+0000", 2024-09-12T07:20:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0025845690000000001, 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026080410000000002, 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63622 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270566+0000", 2024-09-12T07:20:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270566+0000", 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270566+0000", 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.08e-07 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270568+0000", 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9599999999999999e-06 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270575+0000", 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4529999999999999e-06 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270580+0000", 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7849999999999999e-06 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63627 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270589+0000", 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0025612880000000001, 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026420900000000002, 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63627 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270589+0000", 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270589+0000", 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270590+0000", 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.03e-07 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270592+0000", 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.114e-06 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270598+0000", 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5470000000000002e-06 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270603+0000", 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5410000000000002e-06 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63633 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270616+0000", 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0025344009999999999, 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002663286, 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63633 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270616+0000", 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270616+0000", 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270617+0000", 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1600000000000001e-07 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270620+0000", 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.14e-06 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270628+0000", 2024-09-12T07:20:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6230000000000003e-06 2024-09-12T07:20:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270634+0000", 2024-09-12T07:20:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5679999999999999e-06 2024-09-12T07:20:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63637 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270731+0000", 2024-09-12T07:20:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.002419193, 2024-09-12T07:20:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0025971269999999999, 2024-09-12T07:20:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63637 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270731+0000", 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270731+0000", 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270732+0000", 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5600000000000005e-07 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270736+0000", 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270745+0000", 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9600000000000006e-06 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270857+0000", 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011207800000000001 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63639 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270876+0000", 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0022750259999999999, 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0025009699999999999, 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63639 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270876+0000", 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270876+0000", 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270876+0000", 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0500000000000003e-07 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270880+0000", 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.444e-06 2024-09-12T07:20:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270888+0000", 2024-09-12T07:20:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2120000000000003e-06 2024-09-12T07:20:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270894+0000", 2024-09-12T07:20:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4010000000000002e-06 2024-09-12T07:20:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63421 3.5 3:a217c542:::rbd_data.11604a4c57c2.0000000000000054:a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.267524+0000", 2024-09-12T07:20:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.005627015, 2024-09-12T07:20:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059062510000000004, 2024-09-12T07:20:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63421 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267524+0000", 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267524+0000", 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267524+0000", 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5400000000000002e-07 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267527+0000", 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6359999999999998e-06 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267534+0000", 2024-09-12T07:20:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1749999999999999e-06 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267539+0000", 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4659999999999998e-06 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.273170+0000", 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056309719999999997 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.273241+0000", 2024-09-12T07:20:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0302000000000006e-05 2024-09-12T07:20:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63643 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.271436+0000", 2024-09-12T07:20:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0017142850000000001, 2024-09-12T07:20:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020484819999999999, 2024-09-12T07:20:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63643 2024-09-12T07:20:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271436+0000", 2024-09-12T07:20:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271436+0000", 2024-09-12T07:20:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271437+0000", 2024-09-12T07:20:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5499999999999998e-07 2024-09-12T07:20:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271441+0000", 2024-09-12T07:20:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7249999999999999e-06 2024-09-12T07:20:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271453+0000", 2024-09-12T07:20:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.276e-05 2024-09-12T07:20:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271461+0000", 2024-09-12T07:20:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2799999999999998e-06 2024-09-12T07:20:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63430 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.267547+0000", 2024-09-12T07:20:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0056035850000000003, 2024-09-12T07:20:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059880109999999997, 2024-09-12T07:20:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63430 2024-09-12T07:20:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267547+0000", 2024-09-12T07:20:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267547+0000", 2024-09-12T07:20:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267547+0000", 2024-09-12T07:20:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.89e-07 2024-09-12T07:20:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267550+0000", 2024-09-12T07:20:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4650000000000001e-06 2024-09-12T07:20:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267557+0000", 2024-09-12T07:20:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0570000000000004e-06 2024-09-12T07:20:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267562+0000", 2024-09-12T07:20:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1560000000000003e-06 2024-09-12T07:20:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.273537+0000", 2024-09-12T07:20:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059751300000000004 2024-09-12T07:20:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63652 3.1c 3:3ce460d0:::rbd_data.116052cf9e71.000000000000000b:9 [read 32768~16384 out=16384b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.271501+0000", 2024-09-12T07:20:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00164905, 2024-09-12T07:20:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020703610000000002, 2024-09-12T07:20:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63652 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271501+0000", 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271501+0000", 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271502+0000", 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.13e-07 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271505+0000", 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9000000000000002e-06 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271512+0000", 2024-09-12T07:20:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3490000000000003e-06 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271518+0000", 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7749999999999998e-06 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.273426+0000", 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019082649999999999 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.273492+0000", 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5579000000000004e-05 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.273572+0000", 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9880000000000001e-05 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63431 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.267570+0000", 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0055809850000000001, 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0060873350000000001, 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63431 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267570+0000", 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267570+0000", 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267570+0000", 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9700000000000002e-07 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267572+0000", 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2630000000000002e-06 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267579+0000", 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0419999999999997e-06 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267585+0000", 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2820000000000004e-06 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63656 3.1c 3.b06273c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.271527+0000", 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0016232690000000001, 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021754999999999999, 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63656 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271527+0000", 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271527+0000", 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271528+0000", 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2599999999999998e-07 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271533+0000", 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5849999999999999e-06 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271540+0000", 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8759999999999999e-06 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271545+0000", 2024-09-12T07:20:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3349999999999997e-06 2024-09-12T07:20:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63432 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.267592+0000", 2024-09-12T07:20:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0055585560000000001, 2024-09-12T07:20:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061565120000000003, 2024-09-12T07:20:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63432 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267592+0000", 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267592+0000", 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267592+0000", 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267594+0000", 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.995e-06 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267601+0000", 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8759999999999999e-06 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267607+0000", 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7780000000000004e-06 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63658 3.1c 3.b06273c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.271653+0000", 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001497055, 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021410439999999999, 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63658 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271653+0000", 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271653+0000", 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271654+0000", 2024-09-12T07:20:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6400000000000002e-07 2024-09-12T07:20:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271657+0000", 2024-09-12T07:20:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.932e-06 2024-09-12T07:20:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271664+0000", 2024-09-12T07:20:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4880000000000001e-06 2024-09-12T07:20:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271671+0000", 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8190000000000003e-06 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63433 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.267621+0000", 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0055293820000000002, 2024-09-12T07:20:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062199819999999998, 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63433 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267621+0000", 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267621+0000", 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267622+0000", 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2499999999999995e-07 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267624+0000", 2024-09-12T07:20:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6730000000000001e-06 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.267635+0000", 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0909e-05 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.268230+0000", 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00059491100000000005 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63660 3.1c 3.b06273c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.271681+0000", 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001469276, 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0022137110000000002, 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63660 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271681+0000", 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271681+0000", 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271682+0000", 2024-09-12T07:20:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271684+0000", 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3889999999999999e-06 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271696+0000", 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1770000000000001e-05 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271798+0000", 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010189200000000001 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63543 3.5 3.fdcf8ac5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270075+0000", 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0030759250000000002, 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0038668560000000001, 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63543 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270075+0000", 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270075+0000", 2024-09-12T07:20:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270076+0000", 2024-09-12T07:20:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1200000000000001e-07 2024-09-12T07:20:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270080+0000", 2024-09-12T07:20:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0539999999999996e-06 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270089+0000", 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4970000000000007e-06 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270097+0000", 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7030000000000006e-06 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63664 3.1c 3.b06273c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.271831+0000", 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001319073, 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021571260000000001, 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63664 2024-09-12T07:20:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271831+0000", 2024-09-12T07:20:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271831+0000", 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271832+0000", 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1900000000000002e-07 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271835+0000", 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.348e-06 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271853+0000", 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7863000000000002e-05 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271865+0000", 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1314999999999999e-05 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63545 3.5 3.fdcf8ac5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270106+0000", 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0030444030000000002, 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039339040000000002, 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63545 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270106+0000", 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270106+0000", 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270107+0000", 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9299999999999998e-07 2024-09-12T07:20:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270109+0000", 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5399999999999998e-06 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270116+0000", 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9460000000000001e-06 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270121+0000", 2024-09-12T07:20:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1479999999999997e-06 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63667 3.1c 3.b06273c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.271879+0000", 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0012711949999999999, 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00220733, 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "no flag points reached", 2024-09-12T07:20:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63667 2024-09-12T07:20:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271879+0000", 2024-09-12T07:20:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271879+0000", 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271880+0000", 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1600000000000001e-07 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271883+0000", 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.9699999999999999e-06 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.271890+0000", 2024-09-12T07:20:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5619999999999998e-06 2024-09-12T07:20:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63546 3.5 3.fdcf8ac5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270128+0000", 2024-09-12T07:20:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0030227409999999998, 2024-09-12T07:20:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040021099999999997, 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63546 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270128+0000", 2024-09-12T07:20:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270128+0000", 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270128+0000", 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270131+0000", 2024-09-12T07:20:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4499999999999998e-06 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270138+0000", 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5939999999999996e-06 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270145+0000", 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1330000000000001e-06 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63548 3.5 3.fdcf8ac5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270153+0000", 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0029977559999999999, 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004024616, 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63548 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270153+0000", 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270153+0000", 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270153+0000", 2024-09-12T07:20:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.32e-07 2024-09-12T07:20:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270155+0000", 2024-09-12T07:20:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1710000000000001e-06 2024-09-12T07:20:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270162+0000", 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4230000000000002e-06 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270167+0000", 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1630000000000004e-06 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63549 3.5 3.fdcf8ac5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270174+0000", 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0029764710000000001, 2024-09-12T07:20:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040516980000000003, 2024-09-12T07:20:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63549 2024-09-12T07:20:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270174+0000", 2024-09-12T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270174+0000", 2024-09-12T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270175+0000", 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270177+0000", 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.322e-06 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270184+0000", 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9970000000000001e-06 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270189+0000", 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63563 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270287+0000", 2024-09-12T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.002863127, 2024-09-12T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039866770000000001, 2024-09-12T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63563 2024-09-12T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270287+0000", 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270287+0000", 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270288+0000", 2024-09-12T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270290+0000", 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2189999999999998e-06 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270297+0000", 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6599999999999998e-06 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270302+0000", 2024-09-12T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.152e-06 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63565 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270308+0000", 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0028422550000000001, 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040160739999999997, 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63565 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270308+0000", 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270308+0000", 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270309+0000", 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0900000000000002e-07 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270311+0000", 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0509999999999999e-06 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270317+0000", 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2169999999999999e-06 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270322+0000", 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.673e-06 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63566 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270328+0000", 2024-09-12T07:20:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0028229269999999998, 2024-09-12T07:20:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040452209999999999, 2024-09-12T07:20:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63566 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270328+0000", 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270328+0000", 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270328+0000", 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1300000000000001e-07 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270336+0000", 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5800000000000003e-06 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270343+0000", 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3529999999999997e-06 2024-09-12T07:20:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270349+0000", 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2960000000000004e-06 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63568 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270357+0000", 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0027932740000000001, 2024-09-12T07:20:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040698289999999996, 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63568 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270357+0000", 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270357+0000", 2024-09-12T07:20:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270358+0000", 2024-09-12T07:20:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.63e-07 2024-09-12T07:20:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270360+0000", 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1629999999999999e-06 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270366+0000", 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5799999999999997e-06 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270372+0000", 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1880000000000001e-06 2024-09-12T07:20:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63569 3.5 3.42a3e845 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270378+0000", 2024-09-12T07:20:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0027722150000000002, 2024-09-12T07:20:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040991819999999998, 2024-09-12T07:20:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63569 2024-09-12T07:20:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270378+0000", 2024-09-12T07:20:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270378+0000", 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270379+0000", 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9499999999999998e-07 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270381+0000", 2024-09-12T07:20:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1289999999999999e-06 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270389+0000", 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8709999999999997e-06 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270394+0000", 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9740000000000001e-06 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63618 3.5 3.fdcf8ac5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270483+0000", 2024-09-12T07:20:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.002667118, 2024-09-12T07:20:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040497709999999998, 2024-09-12T07:20:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63618 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270483+0000", 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270483+0000", 2024-09-12T07:20:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270484+0000", 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270486+0000", 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9080000000000002e-06 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270492+0000", 2024-09-12T07:20:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3239999999999999e-06 2024-09-12T07:20:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270497+0000", 2024-09-12T07:20:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.904e-06 2024-09-12T07:20:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63619 3.5 3.fdcf8ac5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270503+0000", 2024-09-12T07:20:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.002647599, 2024-09-12T07:20:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040819480000000002, 2024-09-12T07:20:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63619 2024-09-12T07:20:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270503+0000", 2024-09-12T07:20:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270503+0000", 2024-09-12T07:20:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270503+0000", 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270505+0000", 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.917e-06 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270512+0000", 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3609999999999997e-06 2024-09-12T07:20:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270516+0000", 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7310000000000001e-06 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63620 3.5 3.fdcf8ac5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270523+0000", 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0026275980000000001, 2024-09-12T07:20:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0041111280000000004, 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63620 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270523+0000", 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270523+0000", 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270523+0000", 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3500000000000002e-07 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270526+0000", 2024-09-12T07:20:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2800000000000002e-06 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270533+0000", 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1189999999999999e-06 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270538+0000", 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3210000000000004e-06 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:63621 3.5 3.fdcf8ac5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.270545+0000", 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0026055140000000002, 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0041408820000000002, 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 63621 2024-09-12T07:20:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270545+0000", 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270545+0000", 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270545+0000", 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.32e-07 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270548+0000", 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0980000000000001e-06 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270555+0000", 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3900000000000004e-06 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.270560+0000", 2024-09-12T07:20:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:20:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T07:20:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 31 2024-09-12T07:20:03.397 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:03.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:03.399 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:03.400+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:03.499 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:03.500+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:03.602 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:03.604+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:03.605 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:03.605 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:03.617 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:03.701 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:03.703+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:03.802 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:03.804+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:03.805 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.325209589, 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:20:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:20:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:20:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:20:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:20:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:20:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:20:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:20:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:20:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3251827089999999, 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:20:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:20:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:20:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:20:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3251586890000002, 2024-09-12T07:20:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:20:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3251310429999998, 2024-09-12T07:20:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:20:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:20:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:20:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3251075810000001, 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:20:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3250850539999997, 2024-09-12T07:20:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:20:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:20:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3250632509999996, 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:20:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:20:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:20:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:20:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:20:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:20:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:20:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:20:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:20:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:20:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:20:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:20:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:20:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:20:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.318429471, 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:20:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:20:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:20:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:20:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:20:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:20:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3184094630000001, 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3119264309999998, 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:20:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:20:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:20:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:20:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:20:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3118911439999996, 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:20:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3118685110000001, 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3118144560000005, 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:20:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:20:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:20:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.311795515, 2024-09-12T07:20:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:20:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:20:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:20:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:20:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:20:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:20:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:20:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:20:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:20:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:20:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:20:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:20:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:20:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:20:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:20:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.311783578, 2024-09-12T07:20:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:20:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:20:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:20:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:20:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:20:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:20:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:20:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:20:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:20:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:20:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:20:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:20:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:20:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:20:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:20:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.311771963, 2024-09-12T07:20:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:20:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:20:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3047593549999998, 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:20:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3047356399999996, 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:20:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.3046047830000003, 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:20:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.34093399299999999, 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:20:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:20:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:20:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:20:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:20:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:03.906 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:03.907 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-12T07:20:03.908 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:03.908 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:03.909+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:04.008 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:04.010+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:04.106 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:04.107 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:04.109 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:04.110+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:04.117 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:04.154 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.7 to 1 (10000) 2024-09-12T07:20:04.162 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:20:04.172 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-12T07:20:04.172 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:20:04.214 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:04.216+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:04.311 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:04.312+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:04.312 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:04.312 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:04.312 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:04.313 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:04.313 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:04.321 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:04.412 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:04.414+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:04.430 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:04.430 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:04.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:04.430 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:04.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.430 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:20:04.430 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:04.430 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 11.901105167000001, 2024-09-12T07:20:04.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:20:04.430 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:20:04.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0493075010000004, 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.436 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:20:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:20:04.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0491842240000002, 2024-09-12T07:20:04.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:04.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:20:04.444 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0489808329999999, 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.445 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:20:04.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.449 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0488450059999996, 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:20:04.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:20:04.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0485183500000002, 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.454 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:20:04.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:04.458 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0484794620000004, 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:20:04.462 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0481125880000004, 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.463 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:20:04.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.467 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0479777319999997, 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:20:04.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:20:04.471 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0476078409999996, 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:20:04.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.476 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0474176430000002, 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:20:04.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:20:04.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0471975640000002, 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.481 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:20:04.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0470538989999998, 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0468277270000002, 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:04.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.494 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0466273690000003, 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:20:04.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.046219486, 2024-09-12T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:20:04.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0457536039999997, 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.044151394, 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:20:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0436173589999997, 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.513 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:04.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:20:04.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:04.517 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.7874192889999998, 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.518 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.522 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.522 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.522 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.522 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:04.524 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:04.524 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:04.524+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:04.525 INFO:teuthology.orchestra.run.smithi046.stdout:970 read 0x5b6ffcf thru 0x5b71c0f (0x1c41 bytes) 2024-09-12T07:20:04.525 INFO:teuthology.orchestra.run.smithi046.stdout:971 writesame 0x3583adf thru 0x358ffd6 data_size 0x2c(0xc4f8 bytes) 2024-09-12T07:20:04.525 INFO:teuthology.orchestra.run.smithi046.stdout:972 write 0x8a9819 thru 0x8b3390 (0x9b78 bytes) 2024-09-12T07:20:04.525 INFO:teuthology.orchestra.run.smithi046.stdout:973 punch from 0x93ec6ba to 0x93f997c, (0xd2c2 bytes) 2024-09-12T07:20:04.525 INFO:teuthology.orchestra.run.smithi046.stdout:974 punch from 0x57ae1ef to 0x57ba9fa, (0xc80b bytes) 2024-09-12T07:20:04.525 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:04.525 INFO:teuthology.orchestra.run.smithi046.stdout:976 write 0x30bae58 thru 0x30bd6e1 (0x288a bytes) 2024-09-12T07:20:04.525 INFO:teuthology.orchestra.run.smithi046.stdout:977 trunc from 0xa5bc74d to 0x6fc53fa 2024-09-12T07:20:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:978 trunc from 0x6fc53fa to 0x9a678f 2024-09-12T07:20:04.579 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:20:04.580 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:20:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-12T07:20:04.154475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":225766766,"num_objects":214,"num_object_clones":33,"num_object_copies":428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":214,"num_whiteouts":60,"num_read":68675,"num_read_kb":114302,"num_write":1946,"num_write_kb":48550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1124955,"num_keys_recovered":12,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1626,"ondisk_log_size":1626,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":298348,"kb_used_data":80008,"kb_used_omap":10,"kb_used_meta":218165,"kb_avail":660304532,"statfs":{"total":676457349120,"available":676151840768,"internally_reserved":0,"allocated":81928192,"data_stored":85704191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":223401066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":77327036,"num_objects":66,"num_object_clones":9,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":17,"num_read":24288,"num_read_kb":64617,"num_write":491,"num_write_kb":20616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016340"},"pg_stats":[{"pgid":"3.1f","version":"41'22","reported_seq":217,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.201553+0000","last_change":"2024-09-12T07:19:44.455827+0000","last_active":"2024-09-12T07:20:02.201553+0000","last_peered":"2024-09-12T07:20:02.201553+0000","last_clean":"2024-09-12T07:20:02.201553+0000","last_became_active":"2024-09-12T07:19:42.451212+0000","last_became_peered":"2024-09-12T07:19:42.451212+0000","last_unstale":"2024-09-12T07:20:02.201553+0000","last_undegraded":"2024-09-12T07:20:02.201553+0000","last_fullsized":"2024-09-12T07:20:02.201553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:53:10.865460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001023672,"stat_sum":{"num_bytes":6364176,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":151,"num_read_kb":123,"num_write":18,"num_write_kb":215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"41'61","reported_seq":15696,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.467232+0000","last_change":"2024-09-12T07:19:53.514728+0000","last_active":"2024-09-12T07:20:02.467232+0000","last_peered":"2024-09-12T07:20:02.467232+0000","last_clean":"2024-09-12T07:20:02.467232+0000","last_became_active":"2024-09-12T07:19:53.498387+0000","last_became_peered":"2024-09-12T07:19:53.498387+0000","last_unstale":"2024-09-12T07:20:02.467232+0000","last_undegraded":"2024-09-12T07:20:02.467232+0000","last_fullsized":"2024-09-12T07:20:02.467232+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:46.215566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081935099999999996,"stat_sum":{"num_bytes":8614899,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":16280,"num_read_kb":16316,"num_write":59,"num_write_kb":392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":35337,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"41'36","reported_seq":16754,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.153297+0000","last_change":"2024-09-12T07:19:44.464376+0000","last_active":"2024-09-12T07:20:02.153297+0000","last_peered":"2024-09-12T07:20:02.153297+0000","last_clean":"2024-09-12T07:20:02.153297+0000","last_became_active":"2024-09-12T07:19:42.451973+0000","last_became_peered":"2024-09-12T07:19:42.451973+0000","last_unstale":"2024-09-12T07:20:02.153297+0000","last_undegraded":"2024-09-12T07:20:02.153297+0000","last_fullsized":"2024-09-12T07:20:02.153297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:33:04.625322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036949999999999,"stat_sum":{"num_bytes":14940880,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":16685,"num_read_kb":18525,"num_write":36,"num_write_kb":1872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"42'91","reported_seq":392,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.476310+0000","last_change":"2024-09-12T07:19:44.464287+0000","last_active":"2024-09-12T07:20:02.476310+0000","last_peered":"2024-09-12T07:20:02.476310+0000","last_clean":"2024-09-12T07:20:02.476310+0000","last_became_active":"2024-09-12T07:19:42.455338+0000","last_became_peered":"2024-09-12T07:19:42.455338+0000","last_unstale":"2024-09-12T07:20:02.476310+0000","last_undegraded":"2024-09-12T07:20:02.476310+0000","last_fullsized":"2024-09-12T07:20:02.476310+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:55.597955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012347180000000001,"stat_sum":{"num_bytes":8021768,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":584,"num_read_kb":2611,"num_write":130,"num_write_kb":459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"34'38","reported_seq":12595,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.429512+0000","last_change":"2024-09-12T07:19:44.456652+0000","last_active":"2024-09-12T07:20:02.429512+0000","last_peered":"2024-09-12T07:20:02.429512+0000","last_clean":"2024-09-12T07:20:02.429512+0000","last_became_active":"2024-09-12T07:19:42.454990+0000","last_became_peered":"2024-09-12T07:19:42.454990+0000","last_unstale":"2024-09-12T07:20:02.429512+0000","last_undegraded":"2024-09-12T07:20:02.429512+0000","last_fullsized":"2024-09-12T07:20:02.429512+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:48.068380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049866,"stat_sum":{"num_bytes":9038441,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":12536,"num_read_kb":12669,"num_write":34,"num_write_kb":459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"41'27","reported_seq":411,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.473353+0000","last_change":"2024-09-12T07:19:44.459417+0000","last_active":"2024-09-12T07:20:02.473353+0000","last_peered":"2024-09-12T07:20:02.473353+0000","last_clean":"2024-09-12T07:20:02.473353+0000","last_became_active":"2024-09-12T07:19:42.454620+0000","last_became_peered":"2024-09-12T07:19:42.454620+0000","last_unstale":"2024-09-12T07:20:02.473353+0000","last_undegraded":"2024-09-12T07:20:02.473353+0000","last_fullsized":"2024-09-12T07:20:02.473353+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:40:12.977156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090501500000000001,"stat_sum":{"num_bytes":1812024,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":333,"num_read_kb":361,"num_write":23,"num_write_kb":182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"42'186","reported_seq":291,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.430671+0000","last_change":"2024-09-12T07:19:56.126519+0000","last_active":"2024-09-12T07:20:02.430671+0000","last_peered":"2024-09-12T07:20:02.430671+0000","last_clean":"2024-09-12T07:20:02.430671+0000","last_became_active":"2024-09-12T07:19:56.125916+0000","last_became_peered":"2024-09-12T07:19:56.125916+0000","last_unstale":"2024-09-12T07:20:02.430671+0000","last_undegraded":"2024-09-12T07:20:02.430671+0000","last_fullsized":"2024-09-12T07:20:02.430671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:23.647468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113156,"stat_sum":{"num_bytes":8913046,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":842,"num_read_kb":697,"num_write":294,"num_write_kb":1464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":51278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"39'41","reported_seq":188,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.524314+0000","last_change":"2024-09-12T07:19:53.512929+0000","last_active":"2024-09-12T07:20:02.524314+0000","last_peered":"2024-09-12T07:20:02.524314+0000","last_clean":"2024-09-12T07:20:02.524314+0000","last_became_active":"2024-09-12T07:19:53.499240+0000","last_became_peered":"2024-09-12T07:19:53.499240+0000","last_unstale":"2024-09-12T07:20:02.524314+0000","last_undegraded":"2024-09-12T07:20:02.524314+0000","last_fullsized":"2024-09-12T07:20:02.524314+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:02:11.098368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000443294,"stat_sum":{"num_bytes":21012248,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":97,"num_read_kb":703,"num_write":41,"num_write_kb":1005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":74070,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"41'18","reported_seq":99,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.201493+0000","last_change":"2024-09-12T07:19:44.456115+0000","last_active":"2024-09-12T07:20:02.201493+0000","last_peered":"2024-09-12T07:20:02.201493+0000","last_clean":"2024-09-12T07:20:02.201493+0000","last_became_active":"2024-09-12T07:19:42.452617+0000","last_became_peered":"2024-09-12T07:19:42.452617+0000","last_unstale":"2024-09-12T07:20:02.201493+0000","last_undegraded":"2024-09-12T07:20:02.201493+0000","last_fullsized":"2024-09-12T07:20:02.201493+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:25:59.601621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077251,"stat_sum":{"num_bytes":3506176,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":32,"num_read_kb":1066,"num_write":19,"num_write_kb":699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"41'14","reported_seq":75,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.153135+0000","last_change":"2024-09-12T07:19:44.463507+0000","last_active":"2024-09-12T07:20:02.153135+0000","last_peered":"2024-09-12T07:20:02.153135+0000","last_clean":"2024-09-12T07:20:02.153135+0000","last_became_active":"2024-09-12T07:19:42.455210+0000","last_became_peered":"2024-09-12T07:19:42.455210+0000","last_unstale":"2024-09-12T07:20:02.153135+0000","last_undegraded":"2024-09-12T07:20:02.153135+0000","last_fullsized":"2024-09-12T07:20:02.153135+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:57.568628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049753700000000004,"stat_sum":{"num_bytes":3846016,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":4,"num_read_kb":74,"num_write":12,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"42'25","reported_seq":116,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.431894+0000","last_change":"2024-09-12T07:19:53.505519+0000","last_active":"2024-09-12T07:20:02.431894+0000","last_peered":"2024-09-12T07:20:02.431894+0000","last_clean":"2024-09-12T07:20:02.431894+0000","last_became_active":"2024-09-12T07:19:53.500732+0000","last_became_peered":"2024-09-12T07:19:53.500732+0000","last_unstale":"2024-09-12T07:20:02.431894+0000","last_undegraded":"2024-09-12T07:20:02.431894+0000","last_fullsized":"2024-09-12T07:20:02.431894+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:58.842586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012225490000000001,"stat_sum":{"num_bytes":868394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":86,"num_read_kb":62,"num_write":21,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"41'21","reported_seq":215,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.539456+0000","last_change":"2024-09-12T07:19:44.481796+0000","last_active":"2024-09-12T07:20:02.539456+0000","last_peered":"2024-09-12T07:20:02.539456+0000","last_clean":"2024-09-12T07:20:02.539456+0000","last_became_active":"2024-09-12T07:19:42.452277+0000","last_became_peered":"2024-09-12T07:19:42.452277+0000","last_unstale":"2024-09-12T07:20:02.539456+0000","last_undegraded":"2024-09-12T07:20:02.539456+0000","last_fullsized":"2024-09-12T07:20:02.539456+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:02.604213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012485160000000001,"stat_sum":{"num_bytes":17336686,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":171,"num_read_kb":12799,"num_write":29,"num_write_kb":10182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.158446+0000","last_change":"2024-09-12T07:19:32.474512+0000","last_active":"2024-09-12T07:20:02.158446+0000","last_peered":"2024-09-12T07:20:02.158446+0000","last_clean":"2024-09-12T07:20:02.158446+0000","last_became_active":"2024-09-12T07:19:30.219990+0000","last_became_peered":"2024-09-12T07:19:30.219990+0000","last_unstale":"2024-09-12T07:20:02.158446+0000","last_undegraded":"2024-09-12T07:20:02.158446+0000","last_fullsized":"2024-09-12T07:20:02.158446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:32.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046339800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"39'129","reported_seq":189,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.201327+0000","last_change":"2024-09-12T07:19:44.455257+0000","last_active":"2024-09-12T07:20:02.201327+0000","last_peered":"2024-09-12T07:20:02.201327+0000","last_clean":"2024-09-12T07:20:02.201327+0000","last_became_active":"2024-09-12T07:19:42.449730+0000","last_became_peered":"2024-09-12T07:19:42.449730+0000","last_unstale":"2024-09-12T07:20:02.201327+0000","last_undegraded":"2024-09-12T07:20:02.201327+0000","last_fullsized":"2024-09-12T07:20:02.201327+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:23.498748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059259600000000003,"stat_sum":{"num_bytes":30,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":451,"num_read_kb":355,"num_write":183,"num_write_kb":562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":60,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.148866+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:20:02.148866+0000","last_peered":"2024-09-12T07:20:02.148866+0000","last_clean":"2024-09-12T07:20:02.148866+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:20:02.148866+0000","last_undegraded":"2024-09-12T07:20:02.148866+0000","last_fullsized":"2024-09-12T07:20:02.148866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"41'22","reported_seq":100,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.149801+0000","last_change":"2024-09-12T07:19:53.521024+0000","last_active":"2024-09-12T07:20:02.149801+0000","last_peered":"2024-09-12T07:20:02.149801+0000","last_clean":"2024-09-12T07:20:02.149801+0000","last_became_active":"2024-09-12T07:19:53.500929+0000","last_became_peered":"2024-09-12T07:19:53.500929+0000","last_unstale":"2024-09-12T07:20:02.149801+0000","last_undegraded":"2024-09-12T07:20:02.149801+0000","last_fullsized":"2024-09-12T07:20:02.149801+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:56.862515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087343100000000003,"stat_sum":{"num_bytes":12685974,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":9,"num_read_kb":0,"num_write":25,"num_write_kb":1240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":115752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.149820+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:20:02.149820+0000","last_peered":"2024-09-12T07:20:02.149820+0000","last_clean":"2024-09-12T07:20:02.149820+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:20:02.149820+0000","last_undegraded":"2024-09-12T07:20:02.149820+0000","last_fullsized":"2024-09-12T07:20:02.149820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"42'20","reported_seq":190,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.439420+0000","last_change":"2024-09-12T07:19:44.458925+0000","last_active":"2024-09-12T07:20:02.439420+0000","last_peered":"2024-09-12T07:20:02.439420+0000","last_clean":"2024-09-12T07:20:02.439420+0000","last_became_active":"2024-09-12T07:19:42.452704+0000","last_became_peered":"2024-09-12T07:19:42.452704+0000","last_unstale":"2024-09-12T07:20:02.439420+0000","last_undegraded":"2024-09-12T07:20:02.439420+0000","last_fullsized":"2024-09-12T07:20:02.439420+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:15.886369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045577200000000001,"stat_sum":{"num_bytes":1124352,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":346,"num_read_kb":295,"num_write":18,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":60,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.148908+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:20:02.148908+0000","last_peered":"2024-09-12T07:20:02.148908+0000","last_clean":"2024-09-12T07:20:02.148908+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:20:02.148908+0000","last_undegraded":"2024-09-12T07:20:02.148908+0000","last_fullsized":"2024-09-12T07:20:02.148908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"41'21","reported_seq":305,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.525654+0000","last_change":"2024-09-12T07:19:53.503496+0000","last_active":"2024-09-12T07:20:02.525654+0000","last_peered":"2024-09-12T07:20:02.525654+0000","last_clean":"2024-09-12T07:20:02.525654+0000","last_became_active":"2024-09-12T07:19:53.497945+0000","last_became_peered":"2024-09-12T07:19:53.497945+0000","last_unstale":"2024-09-12T07:20:02.525654+0000","last_undegraded":"2024-09-12T07:20:02.525654+0000","last_fullsized":"2024-09-12T07:20:02.525654+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:12:12.070377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015134619999999999,"stat_sum":{"num_bytes":16403680,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":230,"num_read_kb":2811,"num_write":25,"num_write_kb":2164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"41'7","reported_seq":71,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.201491+0000","last_change":"2024-09-12T07:19:44.456465+0000","last_active":"2024-09-12T07:20:02.201491+0000","last_peered":"2024-09-12T07:20:02.201491+0000","last_clean":"2024-09-12T07:20:02.201491+0000","last_became_active":"2024-09-12T07:19:42.454103+0000","last_became_peered":"2024-09-12T07:19:42.454103+0000","last_unstale":"2024-09-12T07:20:02.201491+0000","last_undegraded":"2024-09-12T07:20:02.201491+0000","last_fullsized":"2024-09-12T07:20:02.201491+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:01:40.881332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010980880000000001,"stat_sum":{"num_bytes":1048576,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":9,"num_read_kb":11,"num_write":5,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.153098+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:20:02.153098+0000","last_peered":"2024-09-12T07:20:02.153098+0000","last_clean":"2024-09-12T07:20:02.153098+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:20:02.153098+0000","last_undegraded":"2024-09-12T07:20:02.153098+0000","last_fullsized":"2024-09-12T07:20:02.153098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"41'98","reported_seq":502,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.446281+0000","last_change":"2024-09-12T07:19:44.458868+0000","last_active":"2024-09-12T07:20:02.446281+0000","last_peered":"2024-09-12T07:20:02.446281+0000","last_clean":"2024-09-12T07:20:02.446281+0000","last_became_active":"2024-09-12T07:19:42.451807+0000","last_became_peered":"2024-09-12T07:19:42.451807+0000","last_unstale":"2024-09-12T07:20:02.446281+0000","last_undegraded":"2024-09-12T07:20:02.446281+0000","last_fullsized":"2024-09-12T07:20:02.446281+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:24.806610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048363399999999998,"stat_sum":{"num_bytes":3282746,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":1137,"num_read_kb":973,"num_write":110,"num_write_kb":258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.224110+0000","last_change":"2024-09-12T07:19:53.499895+0000","last_active":"2024-09-12T07:20:02.224110+0000","last_peered":"2024-09-12T07:20:02.224110+0000","last_clean":"2024-09-12T07:20:02.224110+0000","last_became_active":"2024-09-12T07:19:53.499418+0000","last_became_peered":"2024-09-12T07:19:53.499418+0000","last_unstale":"2024-09-12T07:20:02.224110+0000","last_undegraded":"2024-09-12T07:20:02.224110+0000","last_fullsized":"2024-09-12T07:20:02.224110+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:21.122316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"41'53","reported_seq":503,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.459821+0000","last_change":"2024-09-12T07:19:44.455560+0000","last_active":"2024-09-12T07:20:02.459821+0000","last_peered":"2024-09-12T07:20:02.459821+0000","last_clean":"2024-09-12T07:20:02.459821+0000","last_became_active":"2024-09-12T07:19:42.451970+0000","last_became_peered":"2024-09-12T07:19:42.451970+0000","last_unstale":"2024-09-12T07:20:02.459821+0000","last_undegraded":"2024-09-12T07:20:02.459821+0000","last_fullsized":"2024-09-12T07:20:02.459821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:11:23.694124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083352100000000004,"stat_sum":{"num_bytes":3196961,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":1166,"num_read_kb":1044,"num_write":58,"num_write_kb":2081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":60,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.149407+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:20:02.149407+0000","last_peered":"2024-09-12T07:20:02.149407+0000","last_clean":"2024-09-12T07:20:02.149407+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:20:02.149407+0000","last_undegraded":"2024-09-12T07:20:02.149407+0000","last_fullsized":"2024-09-12T07:20:02.149407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"42'55","reported_seq":146,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.157978+0000","last_change":"2024-09-12T07:19:53.538615+0000","last_active":"2024-09-12T07:20:02.157978+0000","last_peered":"2024-09-12T07:20:02.157978+0000","last_clean":"2024-09-12T07:20:02.157978+0000","last_became_active":"2024-09-12T07:19:53.502164+0000","last_became_peered":"2024-09-12T07:19:53.502164+0000","last_unstale":"2024-09-12T07:20:02.157978+0000","last_undegraded":"2024-09-12T07:20:02.157978+0000","last_fullsized":"2024-09-12T07:20:02.157978+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:13:24.314004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089638599999999999,"stat_sum":{"num_bytes":3142058,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":177,"num_read_kb":227,"num_write":78,"num_write_kb":467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":100542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":61,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.153028+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:20:02.153028+0000","last_peered":"2024-09-12T07:20:02.153028+0000","last_clean":"2024-09-12T07:20:02.153028+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:20:02.153028+0000","last_undegraded":"2024-09-12T07:20:02.153028+0000","last_fullsized":"2024-09-12T07:20:02.153028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":151,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.149518+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:20:02.149518+0000","last_peered":"2024-09-12T07:20:02.149518+0000","last_clean":"2024-09-12T07:20:02.149518+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:20:02.149518+0000","last_undegraded":"2024-09-12T07:20:02.149518+0000","last_fullsized":"2024-09-12T07:20:02.149518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"41'31","reported_seq":131,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.149190+0000","last_change":"2024-09-12T07:19:53.521367+0000","last_active":"2024-09-12T07:20:02.149190+0000","last_peered":"2024-09-12T07:20:02.149190+0000","last_clean":"2024-09-12T07:20:02.149190+0000","last_became_active":"2024-09-12T07:19:53.501094+0000","last_became_peered":"2024-09-12T07:19:53.501094+0000","last_unstale":"2024-09-12T07:20:02.149190+0000","last_undegraded":"2024-09-12T07:20:02.149190+0000","last_fullsized":"2024-09-12T07:20:02.149190+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:47.486124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047491800000000001,"stat_sum":{"num_bytes":3860875,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":101,"num_read_kb":385,"num_write":40,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":128643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"39'84","reported_seq":653,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.525562+0000","last_change":"2024-09-12T07:19:44.469508+0000","last_active":"2024-09-12T07:20:02.525562+0000","last_peered":"2024-09-12T07:20:02.525562+0000","last_clean":"2024-09-12T07:20:02.525562+0000","last_became_active":"2024-09-12T07:19:42.450230+0000","last_became_peered":"2024-09-12T07:19:42.450230+0000","last_unstale":"2024-09-12T07:20:02.525562+0000","last_undegraded":"2024-09-12T07:20:02.525562+0000","last_fullsized":"2024-09-12T07:20:02.525562+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:04.100968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076460899999999997,"stat_sum":{"num_bytes":20347489,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":1387,"num_read_kb":2442,"num_write":79,"num_write_kb":1217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"42'18","reported_seq":76,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.158496+0000","last_change":"2024-09-12T07:19:44.480981+0000","last_active":"2024-09-12T07:20:02.158496+0000","last_peered":"2024-09-12T07:20:02.158496+0000","last_clean":"2024-09-12T07:20:02.158496+0000","last_became_active":"2024-09-12T07:19:42.452637+0000","last_became_peered":"2024-09-12T07:19:42.452637+0000","last_unstale":"2024-09-12T07:20:02.158496+0000","last_undegraded":"2024-09-12T07:20:02.158496+0000","last_fullsized":"2024-09-12T07:20:02.158496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:25.797127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000680892,"stat_sum":{"num_bytes":884736,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":11,"num_read_kb":8,"num_write":17,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"41'17","reported_seq":152,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.480650+0000","last_change":"2024-09-12T07:19:53.527716+0000","last_active":"2024-09-12T07:20:02.480650+0000","last_peered":"2024-09-12T07:20:02.480650+0000","last_clean":"2024-09-12T07:20:02.480650+0000","last_became_active":"2024-09-12T07:19:53.500926+0000","last_became_peered":"2024-09-12T07:19:53.500926+0000","last_unstale":"2024-09-12T07:20:02.480650+0000","last_undegraded":"2024-09-12T07:20:02.480650+0000","last_fullsized":"2024-09-12T07:20:02.480650+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:14:50.604581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054644100000000005,"stat_sum":{"num_bytes":2514028,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":102,"num_read_kb":549,"num_write":15,"num_write_kb":1064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":532572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"41'16","reported_seq":159,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.518064+0000","last_change":"2024-09-12T07:19:53.545055+0000","last_active":"2024-09-12T07:20:02.518064+0000","last_peered":"2024-09-12T07:20:02.518064+0000","last_clean":"2024-09-12T07:20:02.518064+0000","last_became_active":"2024-09-12T07:19:53.502246+0000","last_became_peered":"2024-09-12T07:19:53.502246+0000","last_unstale":"2024-09-12T07:20:02.518064+0000","last_undegraded":"2024-09-12T07:20:02.518064+0000","last_fullsized":"2024-09-12T07:20:02.518064+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:06.543538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050446300000000003,"stat_sum":{"num_bytes":9884978,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":83,"num_read_kb":651,"num_write":15,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":60385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"42'34","reported_seq":193,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.378730+0000","last_change":"2024-09-12T07:19:44.482470+0000","last_active":"2024-09-12T07:20:02.378730+0000","last_peered":"2024-09-12T07:20:02.378730+0000","last_clean":"2024-09-12T07:20:02.378730+0000","last_became_active":"2024-09-12T07:19:42.450577+0000","last_became_peered":"2024-09-12T07:19:42.450577+0000","last_unstale":"2024-09-12T07:20:02.378730+0000","last_undegraded":"2024-09-12T07:20:02.378730+0000","last_fullsized":"2024-09-12T07:20:02.378730+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:34.810513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317836,"stat_sum":{"num_bytes":3325952,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":287,"num_read_kb":1244,"num_write":31,"num_write_kb":1097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"41'91","reported_seq":616,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.473726+0000","last_change":"2024-09-12T07:19:44.481850+0000","last_active":"2024-09-12T07:20:02.473726+0000","last_peered":"2024-09-12T07:20:02.473726+0000","last_clean":"2024-09-12T07:20:02.473726+0000","last_became_active":"2024-09-12T07:19:42.452236+0000","last_became_peered":"2024-09-12T07:19:42.452236+0000","last_unstale":"2024-09-12T07:20:02.473726+0000","last_undegraded":"2024-09-12T07:20:02.473726+0000","last_fullsized":"2024-09-12T07:20:02.473726+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:43:51.412978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012939710000000001,"stat_sum":{"num_bytes":1260844,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":1409,"num_read_kb":1303,"num_write":83,"num_write_kb":659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"42'164","reported_seq":388,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.453314+0000","last_change":"2024-09-12T07:19:44.482179+0000","last_active":"2024-09-12T07:20:02.453314+0000","last_peered":"2024-09-12T07:20:02.453314+0000","last_clean":"2024-09-12T07:20:02.453314+0000","last_became_active":"2024-09-12T07:19:42.452155+0000","last_became_peered":"2024-09-12T07:19:42.452155+0000","last_unstale":"2024-09-12T07:20:02.453314+0000","last_undegraded":"2024-09-12T07:20:02.453314+0000","last_fullsized":"2024-09-12T07:20:02.453314+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:51.339782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011884759999999999,"stat_sum":{"num_bytes":957551,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":1165,"num_read_kb":948,"num_write":230,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"41'6","reported_seq":64,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.158380+0000","last_change":"2024-09-12T07:19:44.481020+0000","last_active":"2024-09-12T07:20:02.158380+0000","last_peered":"2024-09-12T07:20:02.158380+0000","last_clean":"2024-09-12T07:20:02.158380+0000","last_became_active":"2024-09-12T07:19:42.450535+0000","last_became_peered":"2024-09-12T07:19:42.450535+0000","last_unstale":"2024-09-12T07:20:02.158380+0000","last_undegraded":"2024-09-12T07:20:02.158380+0000","last_fullsized":"2024-09-12T07:20:02.158380+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:53:45.505347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073032100000000003,"stat_sum":{"num_bytes":1048576,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"41'36","reported_seq":186,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.535579+0000","last_change":"2024-09-12T07:19:44.481275+0000","last_active":"2024-09-12T07:20:02.535579+0000","last_peered":"2024-09-12T07:20:02.535579+0000","last_clean":"2024-09-12T07:20:02.535579+0000","last_became_active":"2024-09-12T07:19:42.452104+0000","last_became_peered":"2024-09-12T07:19:42.452104+0000","last_unstale":"2024-09-12T07:20:02.535579+0000","last_undegraded":"2024-09-12T07:20:02.535579+0000","last_fullsized":"2024-09-12T07:20:02.535579+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:25.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079083699999999996,"stat_sum":{"num_bytes":11510788,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":109,"num_read_kb":9964,"num_write":35,"num_write_kb":7165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"40'8","reported_seq":84,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.201353+0000","last_change":"2024-09-12T07:19:44.456086+0000","last_active":"2024-09-12T07:20:02.201353+0000","last_peered":"2024-09-12T07:20:02.201353+0000","last_clean":"2024-09-12T07:20:02.201353+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:20:02.201353+0000","last_undegraded":"2024-09-12T07:20:02.201353+0000","last_fullsized":"2024-09-12T07:20:02.201353+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:01:49.831173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060746,"stat_sum":{"num_bytes":6829634,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":20,"num_read_kb":1895,"num_write":13,"num_write_kb":1694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"40'61","reported_seq":12683,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.538545+0000","last_change":"2024-09-12T07:19:44.482564+0000","last_active":"2024-09-12T07:20:02.538545+0000","last_peered":"2024-09-12T07:20:02.538545+0000","last_clean":"2024-09-12T07:20:02.538545+0000","last_became_active":"2024-09-12T07:19:42.456127+0000","last_became_peered":"2024-09-12T07:19:42.456127+0000","last_unstale":"2024-09-12T07:20:02.538545+0000","last_undegraded":"2024-09-12T07:20:02.538545+0000","last_fullsized":"2024-09-12T07:20:02.538545+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:30:22.533631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793039999999999,"stat_sum":{"num_bytes":17591797,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":12606,"num_read_kb":23136,"num_write":58,"num_write_kb":7976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":225176379,"num_objects":210,"num_object_clones":33,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":60,"num_read":68609,"num_read_kb":114247,"num_write":1843,"num_write_kb":46920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1124955,"num_keys_recovered":12,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":78729216,"data_stored":84842066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1551,"ondisk_log_size":1551,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542155,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60056,"kb_used_data":28868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311784,"statfs":{"total":96636764160,"available":96575266816,"internally_reserved":0,"allocated":29560832,"data_stored":29676342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28736,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343104,"statfs":{"total":96636764160,"available":96607338496,"internally_reserved":0,"allocated":1744896,"data_stored":1466504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":5812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":5951488,"data_stored":8238770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61876,"kb_used_data":30688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309964,"statfs":{"total":96636764160,"available":96573403136,"internally_reserved":0,"allocated":31424512,"data_stored":33034797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":60129542155,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542155,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34136,"kb_used_data":2944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337704,"statfs":{"total":96636764160,"available":96601808896,"internally_reserved":0,"allocated":3014656,"data_stored":2706153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37244,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334596,"statfs":{"total":96636764160,"available":96598626304,"internally_reserved":0,"allocated":1937408,"data_stored":1559130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43464,"kb_used_data":8100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328376,"statfs":{"total":96636764160,"available":96592257024,"internally_reserved":0,"allocated":8294400,"data_stored":9022495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7241728,"data_stored":8240643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1478656,"data_stored":1367703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2555904,"data_stored":2514669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1232896,"data_stored":1677356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":30953472,"data_stored":32837162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5488640,"data_stored":8047267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1273856,"data_stored":1268869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28504064,"data_stored":28888397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:20:04.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:20:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:980 read 0x5f88e1 thru 0x60609d (0xd7bd bytes) 2024-09-12T07:20:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:981 write 0xd92c773 thru 0xd93a645 (0xded3 bytes) 2024-09-12T07:20:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:984 write 0x425b763 thru 0x4264906 (0x91a4 bytes) 2024-09-12T07:20:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:985 write 0x5cb8b43 thru 0x5cc2f26 (0xa3e4 bytes) 2024-09-12T07:20:04.624 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:04.625+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:986 punch from 0x1d8ef5f to 0x1d936b0, (0x4751 bytes) 2024-09-12T07:20:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:989 write 0xd79aa5a thru 0xd79e3f4 (0x399b bytes) 2024-09-12T07:20:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:990 writesame 0x4a773e9 thru 0x4a85513 data_size 0x1(0xe12b bytes) 2024-09-12T07:20:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:994 trunc from 0xd93a646 to 0x3eddc51 2024-09-12T07:20:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:995 write 0x10b43b8 thru 0x10c00c2 (0xbd0b bytes) 2024-09-12T07:20:04.679 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:04.679 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:04.679 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:04.679 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:04.682 INFO:teuthology.orchestra.run.smithi046.stdout:996 trunc from 0x3eddc51 to 0x309fa4b 2024-09-12T07:20:04.689 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:04.702 INFO:teuthology.orchestra.run.smithi046.stdout:998 write 0x6e18d3e thru 0x6e219c2 (0x8c85 bytes) 2024-09-12T07:20:04.706 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:04.706 INFO:teuthology.orchestra.run.smithi046.stdout:1000 write 0x5af0e8e thru 0x5af5ce2 (0x4e55 bytes) 2024-09-12T07:20:04.708 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:04.720 INFO:teuthology.orchestra.run.smithi046.stdout:1003 write 0xcbf4df0 thru 0xcbfa9ff (0x5c10 bytes) 2024-09-12T07:20:04.725 INFO:teuthology.orchestra.run.smithi046.stdout:1004 read 0x75eab12 thru 0x75f11c1 (0x66b0 bytes) 2024-09-12T07:20:04.725 INFO:teuthology.orchestra.run.smithi046.stdout:1006 write 0x81170f2 thru 0x811854c (0x145b bytes) 2024-09-12T07:20:04.725 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:04.726+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:04.733 INFO:teuthology.orchestra.run.smithi046.stdout:1008 write 0xd3e7821 thru 0xd3f5a86 (0xe266 bytes) 2024-09-12T07:20:04.737 INFO:teuthology.orchestra.run.smithi046.stdout:1011 write 0x77f2700 thru 0x7800304 (0xdc05 bytes) 2024-09-12T07:20:04.741 INFO:teuthology.orchestra.run.smithi046.stdout:1013 write 0x6bd56a2 thru 0x6bd77c1 (0x2120 bytes) 2024-09-12T07:20:04.743 INFO:teuthology.orchestra.run.smithi046.stdout:1014 write 0xa8e8be2 thru 0xa8f2232 (0x9651 bytes) 2024-09-12T07:20:04.747 INFO:teuthology.orchestra.run.smithi046.stdout:1015 write 0x8003994 thru 0x8005586 (0x1bf3 bytes) 2024-09-12T07:20:04.750 INFO:teuthology.orchestra.run.smithi046.stdout:1016 write 0xaefe577 thru 0xaf09e6f (0xb8f9 bytes) 2024-09-12T07:20:04.753 INFO:teuthology.orchestra.run.smithi046.stdout:1017 writesame 0x7b2b95a thru 0x7b32ac3 data_size 0x9(0x716a bytes) 2024-09-12T07:20:04.762 INFO:teuthology.orchestra.run.smithi046.stdout:1018 writesame 0xd421e90 thru 0xd430812 data_size 0x1(0xe983 bytes) 2024-09-12T07:20:04.763 INFO:teuthology.orchestra.run.smithi046.stdout:1021 read 0xf5aa2a thru 0xf69b06 (0xf0dd bytes) 2024-09-12T07:20:04.763 INFO:teuthology.orchestra.run.smithi046.stdout:1022 writesame 0xbded6a3 thru 0xbdf2902 data_size 0x2(0x5260 bytes) 2024-09-12T07:20:04.768 INFO:teuthology.orchestra.run.smithi046.stdout:1025 writesame 0x32136ce thru 0x321b263 data_size 0x3(0x7b96 bytes) 2024-09-12T07:20:04.780 INFO:teuthology.orchestra.run.smithi046.stdout:1026 write 0xd43d1b9 thru 0xd43d889 (0x6d1 bytes) 2024-09-12T07:20:04.783 INFO:teuthology.orchestra.run.smithi046.stdout:1027 punch from 0x8f66a6c to 0x8f7312e, (0xc6c2 bytes) 2024-09-12T07:20:04.796 INFO:teuthology.orchestra.run.smithi046.stdout:1028 trunc from 0xd43d88a to 0x11d8836 2024-09-12T07:20:04.798 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:20:04.810 INFO:teuthology.orchestra.run.smithi046.stdout:1029 read 0xd1062a thru 0xd118c3 (0x129a bytes) 2024-09-12T07:20:04.822 INFO:teuthology.orchestra.run.smithi046.stdout:1030 trunc from 0x11d8836 to 0xb36fe39 2024-09-12T07:20:04.825 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:04.827+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:04.828 INFO:teuthology.orchestra.run.smithi046.stdout:1033 read 0x27aff88 thru 0x27ba708 (0xa781 bytes) 2024-09-12T07:20:04.828 INFO:teuthology.orchestra.run.smithi046.stdout:1035 write 0x4378835 thru 0x437a39c (0x1b68 bytes) 2024-09-12T07:20:04.831 INFO:teuthology.orchestra.run.smithi046.stdout:1036 read 0x1432339 thru 0x1434ffa (0x2cc2 bytes) 2024-09-12T07:20:04.831 INFO:teuthology.orchestra.run.smithi046.stdout:1037 read 0x739d47e thru 0x739f8cc (0x244f bytes) 2024-09-12T07:20:04.832 INFO:teuthology.orchestra.run.smithi046.stdout:1039 write 0x48d0bca thru 0x48d1a54 (0xe8b bytes) 2024-09-12T07:20:04.834 INFO:teuthology.orchestra.run.smithi046.stdout:1040 write 0x91dc9e4 thru 0x91e355e (0x6b7b bytes) 2024-09-12T07:20:04.839 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:04.840 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:04.840 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:04.841 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:04.841 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:04.841 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:04.847 INFO:teuthology.orchestra.run.smithi046.stdout:1043 writesame 0xd50da5c thru 0xd516952 data_size 0x1(0x8ef7 bytes) 2024-09-12T07:20:04.847 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:04.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:04.863 INFO:teuthology.orchestra.run.smithi046.stdout:1046 trunc from 0xd516953 to 0x1bfd58d 2024-09-12T07:20:04.884 INFO:teuthology.orchestra.run.smithi046.stdout:1047 trunc from 0x1bfd58d to 0xadf68da 2024-09-12T07:20:04.890 INFO:teuthology.orchestra.run.smithi046.stdout:1048 write 0x79dce5c thru 0x79e7467 (0xa60c bytes) 2024-09-12T07:20:04.895 INFO:teuthology.orchestra.run.smithi046.stdout:1049 writesame 0xada1d45 thru 0xada72b1 data_size 0x1(0x556d bytes) 2024-09-12T07:20:04.912 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:04.919 INFO:teuthology.orchestra.run.smithi046.stdout:1051 write 0xc41a378 thru 0xc4227d2 (0x845b bytes) 2024-09-12T07:20:04.924 INFO:teuthology.orchestra.run.smithi046.stdout:1055 clone 10 order 21 su 8192 sc 2 2024-09-12T07:20:04.926 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:04.927+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:04.948 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:20:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:20:04.949 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:04.960 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-12T07:20:04.154475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":225766766,"num_objects":214,"num_object_clones":33,"num_object_copies":428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":214,"num_whiteouts":60,"num_read":68675,"num_read_kb":114302,"num_write":1946,"num_write_kb":48550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1124955,"num_keys_recovered":12,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1626,"ondisk_log_size":1626,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":298348,"kb_used_data":80008,"kb_used_omap":10,"kb_used_meta":218165,"kb_avail":660304532,"statfs":{"total":676457349120,"available":676151840768,"internally_reserved":0,"allocated":81928192,"data_stored":85704191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":223401066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":77327036,"num_objects":66,"num_object_clones":9,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":17,"num_read":24288,"num_read_kb":64617,"num_write":491,"num_write_kb":20616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016340"},"pg_stats":[{"pgid":"3.1f","version":"41'22","reported_seq":217,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.201553+0000","last_change":"2024-09-12T07:19:44.455827+0000","last_active":"2024-09-12T07:20:02.201553+0000","last_peered":"2024-09-12T07:20:02.201553+0000","last_clean":"2024-09-12T07:20:02.201553+0000","last_became_active":"2024-09-12T07:19:42.451212+0000","last_became_peered":"2024-09-12T07:19:42.451212+0000","last_unstale":"2024-09-12T07:20:02.201553+0000","last_undegraded":"2024-09-12T07:20:02.201553+0000","last_fullsized":"2024-09-12T07:20:02.201553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:53:10.865460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001023672,"stat_sum":{"num_bytes":6364176,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":151,"num_read_kb":123,"num_write":18,"num_write_kb":215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"41'61","reported_seq":15696,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.467232+0000","last_change":"2024-09-12T07:19:53.514728+0000","last_active":"2024-09-12T07:20:02.467232+0000","last_peered":"2024-09-12T07:20:02.467232+0000","last_clean":"2024-09-12T07:20:02.467232+0000","last_became_active":"2024-09-12T07:19:53.498387+0000","last_became_peered":"2024-09-12T07:19:53.498387+0000","last_unstale":"2024-09-12T07:20:02.467232+0000","last_undegraded":"2024-09-12T07:20:02.467232+0000","last_fullsized":"2024-09-12T07:20:02.467232+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:46.215566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081935099999999996,"stat_sum":{"num_bytes":8614899,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":16280,"num_read_kb":16316,"num_write":59,"num_write_kb":392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":35337,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"41'36","reported_seq":16754,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.153297+0000","last_change":"2024-09-12T07:19:44.464376+0000","last_active":"2024-09-12T07:20:02.153297+0000","last_peered":"2024-09-12T07:20:02.153297+0000","last_clean":"2024-09-12T07:20:02.153297+0000","last_became_active":"2024-09-12T07:19:42.451973+0000","last_became_peered":"2024-09-12T07:19:42.451973+0000","last_unstale":"2024-09-12T07:20:02.153297+0000","last_undegraded":"2024-09-12T07:20:02.153297+0000","last_fullsized":"2024-09-12T07:20:02.153297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:33:04.625322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036949999999999,"stat_sum":{"num_bytes":14940880,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":16685,"num_read_kb":18525,"num_write":36,"num_write_kb":1872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"42'91","reported_seq":392,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.476310+0000","last_change":"2024-09-12T07:19:44.464287+0000","last_active":"2024-09-12T07:20:02.476310+0000","last_peered":"2024-09-12T07:20:02.476310+0000","last_clean":"2024-09-12T07:20:02.476310+0000","last_became_active":"2024-09-12T07:19:42.455338+0000","last_became_peered":"2024-09-12T07:19:42.455338+0000","last_unstale":"2024-09-12T07:20:02.476310+0000","last_undegraded":"2024-09-12T07:20:02.476310+0000","last_fullsized":"2024-09-12T07:20:02.476310+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:55.597955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012347180000000001,"stat_sum":{"num_bytes":8021768,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":584,"num_read_kb":2611,"num_write":130,"num_write_kb":459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"34'38","reported_seq":12595,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.429512+0000","last_change":"2024-09-12T07:19:44.456652+0000","last_active":"2024-09-12T07:20:02.429512+0000","last_peered":"2024-09-12T07:20:02.429512+0000","last_clean":"2024-09-12T07:20:02.429512+0000","last_became_active":"2024-09-12T07:19:42.454990+0000","last_became_peered":"2024-09-12T07:19:42.454990+0000","last_unstale":"2024-09-12T07:20:02.429512+0000","last_undegraded":"2024-09-12T07:20:02.429512+0000","last_fullsized":"2024-09-12T07:20:02.429512+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:48.068380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049866,"stat_sum":{"num_bytes":9038441,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":12536,"num_read_kb":12669,"num_write":34,"num_write_kb":459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"41'27","reported_seq":411,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.473353+0000","last_change":"2024-09-12T07:19:44.459417+0000","last_active":"2024-09-12T07:20:02.473353+0000","last_peered":"2024-09-12T07:20:02.473353+0000","last_clean":"2024-09-12T07:20:02.473353+0000","last_became_active":"2024-09-12T07:19:42.454620+0000","last_became_peered":"2024-09-12T07:19:42.454620+0000","last_unstale":"2024-09-12T07:20:02.473353+0000","last_undegraded":"2024-09-12T07:20:02.473353+0000","last_fullsized":"2024-09-12T07:20:02.473353+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:40:12.977156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090501500000000001,"stat_sum":{"num_bytes":1812024,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":333,"num_read_kb":361,"num_write":23,"num_write_kb":182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"42'186","reported_seq":291,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.430671+0000","last_change":"2024-09-12T07:19:56.126519+0000","last_active":"2024-09-12T07:20:02.430671+0000","last_peered":"2024-09-12T07:20:02.430671+0000","last_clean":"2024-09-12T07:20:02.430671+0000","last_became_active":"2024-09-12T07:19:56.125916+0000","last_became_peered":"2024-09-12T07:19:56.125916+0000","last_unstale":"2024-09-12T07:20:02.430671+0000","last_undegraded":"2024-09-12T07:20:02.430671+0000","last_fullsized":"2024-09-12T07:20:02.430671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:23.647468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113156,"stat_sum":{"num_bytes":8913046,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":842,"num_read_kb":697,"num_write":294,"num_write_kb":1464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":51278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"39'41","reported_seq":188,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.524314+0000","last_change":"2024-09-12T07:19:53.512929+0000","last_active":"2024-09-12T07:20:02.524314+0000","last_peered":"2024-09-12T07:20:02.524314+0000","last_clean":"2024-09-12T07:20:02.524314+0000","last_became_active":"2024-09-12T07:19:53.499240+0000","last_became_peered":"2024-09-12T07:19:53.499240+0000","last_unstale":"2024-09-12T07:20:02.524314+0000","last_undegraded":"2024-09-12T07:20:02.524314+0000","last_fullsized":"2024-09-12T07:20:02.524314+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:02:11.098368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000443294,"stat_sum":{"num_bytes":21012248,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":97,"num_read_kb":703,"num_write":41,"num_write_kb":1005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":74070,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"41'18","reported_seq":99,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.201493+0000","last_change":"2024-09-12T07:19:44.456115+0000","last_active":"2024-09-12T07:20:02.201493+0000","last_peered":"2024-09-12T07:20:02.201493+0000","last_clean":"2024-09-12T07:20:02.201493+0000","last_became_active":"2024-09-12T07:19:42.452617+0000","last_became_peered":"2024-09-12T07:19:42.452617+0000","last_unstale":"2024-09-12T07:20:02.201493+0000","last_undegraded":"2024-09-12T07:20:02.201493+0000","last_fullsized":"2024-09-12T07:20:02.201493+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:25:59.601621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077251,"stat_sum":{"num_bytes":3506176,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":32,"num_read_kb":1066,"num_write":19,"num_write_kb":699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"41'14","reported_seq":75,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.153135+0000","last_change":"2024-09-12T07:19:44.463507+0000","last_active":"2024-09-12T07:20:02.153135+0000","last_peered":"2024-09-12T07:20:02.153135+0000","last_clean":"2024-09-12T07:20:02.153135+0000","last_became_active":"2024-09-12T07:19:42.455210+0000","last_became_peered":"2024-09-12T07:19:42.455210+0000","last_unstale":"2024-09-12T07:20:02.153135+0000","last_undegraded":"2024-09-12T07:20:02.153135+0000","last_fullsized":"2024-09-12T07:20:02.153135+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:57.568628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049753700000000004,"stat_sum":{"num_bytes":3846016,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":4,"num_read_kb":74,"num_write":12,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"42'25","reported_seq":116,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.431894+0000","last_change":"2024-09-12T07:19:53.505519+0000","last_active":"2024-09-12T07:20:02.431894+0000","last_peered":"2024-09-12T07:20:02.431894+0000","last_clean":"2024-09-12T07:20:02.431894+0000","last_became_active":"2024-09-12T07:19:53.500732+0000","last_became_peered":"2024-09-12T07:19:53.500732+0000","last_unstale":"2024-09-12T07:20:02.431894+0000","last_undegraded":"2024-09-12T07:20:02.431894+0000","last_fullsized":"2024-09-12T07:20:02.431894+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:58.842586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012225490000000001,"stat_sum":{"num_bytes":868394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":86,"num_read_kb":62,"num_write":21,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"41'21","reported_seq":215,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.539456+0000","last_change":"2024-09-12T07:19:44.481796+0000","last_active":"2024-09-12T07:20:02.539456+0000","last_peered":"2024-09-12T07:20:02.539456+0000","last_clean":"2024-09-12T07:20:02.539456+0000","last_became_active":"2024-09-12T07:19:42.452277+0000","last_became_peered":"2024-09-12T07:19:42.452277+0000","last_unstale":"2024-09-12T07:20:02.539456+0000","last_undegraded":"2024-09-12T07:20:02.539456+0000","last_fullsized":"2024-09-12T07:20:02.539456+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:02.604213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012485160000000001,"stat_sum":{"num_bytes":17336686,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":171,"num_read_kb":12799,"num_write":29,"num_write_kb":10182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.158446+0000","last_change":"2024-09-12T07:19:32.474512+0000","last_active":"2024-09-12T07:20:02.158446+0000","last_peered":"2024-09-12T07:20:02.158446+0000","last_clean":"2024-09-12T07:20:02.158446+0000","last_became_active":"2024-09-12T07:19:30.219990+0000","last_became_peered":"2024-09-12T07:19:30.219990+0000","last_unstale":"2024-09-12T07:20:02.158446+0000","last_undegraded":"2024-09-12T07:20:02.158446+0000","last_fullsized":"2024-09-12T07:20:02.158446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:32.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046339800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"39'129","reported_seq":189,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.201327+0000","last_change":"2024-09-12T07:19:44.455257+0000","last_active":"2024-09-12T07:20:02.201327+0000","last_peered":"2024-09-12T07:20:02.201327+0000","last_clean":"2024-09-12T07:20:02.201327+0000","last_became_active":"2024-09-12T07:19:42.449730+0000","last_became_peered":"2024-09-12T07:19:42.449730+0000","last_unstale":"2024-09-12T07:20:02.201327+0000","last_undegraded":"2024-09-12T07:20:02.201327+0000","last_fullsized":"2024-09-12T07:20:02.201327+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:23.498748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059259600000000003,"stat_sum":{"num_bytes":30,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":451,"num_read_kb":355,"num_write":183,"num_write_kb":562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":60,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.148866+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:20:02.148866+0000","last_peered":"2024-09-12T07:20:02.148866+0000","last_clean":"2024-09-12T07:20:02.148866+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:20:02.148866+0000","last_undegraded":"2024-09-12T07:20:02.148866+0000","last_fullsized":"2024-09-12T07:20:02.148866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"41'22","reported_seq":100,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.149801+0000","last_change":"2024-09-12T07:19:53.521024+0000","last_active":"2024-09-12T07:20:02.149801+0000","last_peered":"2024-09-12T07:20:02.149801+0000","last_clean":"2024-09-12T07:20:02.149801+0000","last_became_active":"2024-09-12T07:19:53.500929+0000","last_became_peered":"2024-09-12T07:19:53.500929+0000","last_unstale":"2024-09-12T07:20:02.149801+0000","last_undegraded":"2024-09-12T07:20:02.149801+0000","last_fullsized":"2024-09-12T07:20:02.149801+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:56.862515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087343100000000003,"stat_sum":{"num_bytes":12685974,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":9,"num_read_kb":0,"num_write":25,"num_write_kb":1240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":115752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.149820+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:20:02.149820+0000","last_peered":"2024-09-12T07:20:02.149820+0000","last_clean":"2024-09-12T07:20:02.149820+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:20:02.149820+0000","last_undegraded":"2024-09-12T07:20:02.149820+0000","last_fullsized":"2024-09-12T07:20:02.149820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"42'20","reported_seq":190,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.439420+0000","last_change":"2024-09-12T07:19:44.458925+0000","last_active":"2024-09-12T07:20:02.439420+0000","last_peered":"2024-09-12T07:20:02.439420+0000","last_clean":"2024-09-12T07:20:02.439420+0000","last_became_active":"2024-09-12T07:19:42.452704+0000","last_became_peered":"2024-09-12T07:19:42.452704+0000","last_unstale":"2024-09-12T07:20:02.439420+0000","last_undegraded":"2024-09-12T07:20:02.439420+0000","last_fullsized":"2024-09-12T07:20:02.439420+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:15.886369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045577200000000001,"stat_sum":{"num_bytes":1124352,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":346,"num_read_kb":295,"num_write":18,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":60,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.148908+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:20:02.148908+0000","last_peered":"2024-09-12T07:20:02.148908+0000","last_clean":"2024-09-12T07:20:02.148908+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:20:02.148908+0000","last_undegraded":"2024-09-12T07:20:02.148908+0000","last_fullsized":"2024-09-12T07:20:02.148908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"41'21","reported_seq":305,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.525654+0000","last_change":"2024-09-12T07:19:53.503496+0000","last_active":"2024-09-12T07:20:02.525654+0000","last_peered":"2024-09-12T07:20:02.525654+0000","last_clean":"2024-09-12T07:20:02.525654+0000","last_became_active":"2024-09-12T07:19:53.497945+0000","last_became_peered":"2024-09-12T07:19:53.497945+0000","last_unstale":"2024-09-12T07:20:02.525654+0000","last_undegraded":"2024-09-12T07:20:02.525654+0000","last_fullsized":"2024-09-12T07:20:02.525654+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:12:12.070377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015134619999999999,"stat_sum":{"num_bytes":16403680,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":230,"num_read_kb":2811,"num_write":25,"num_write_kb":2164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"41'7","reported_seq":71,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.201491+0000","last_change":"2024-09-12T07:19:44.456465+0000","last_active":"2024-09-12T07:20:02.201491+0000","last_peered":"2024-09-12T07:20:02.201491+0000","last_clean":"2024-09-12T07:20:02.201491+0000","last_became_active":"2024-09-12T07:19:42.454103+0000","last_became_peered":"2024-09-12T07:19:42.454103+0000","last_unstale":"2024-09-12T07:20:02.201491+0000","last_undegraded":"2024-09-12T07:20:02.201491+0000","last_fullsized":"2024-09-12T07:20:02.201491+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:01:40.881332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010980880000000001,"stat_sum":{"num_bytes":1048576,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":9,"num_read_kb":11,"num_write":5,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.153098+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:20:02.153098+0000","last_peered":"2024-09-12T07:20:02.153098+0000","last_clean":"2024-09-12T07:20:02.153098+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:20:02.153098+0000","last_undegraded":"2024-09-12T07:20:02.153098+0000","last_fullsized":"2024-09-12T07:20:02.153098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"41'98","reported_seq":502,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.446281+0000","last_change":"2024-09-12T07:19:44.458868+0000","last_active":"2024-09-12T07:20:02.446281+0000","last_peered":"2024-09-12T07:20:02.446281+0000","last_clean":"2024-09-12T07:20:02.446281+0000","last_became_active":"2024-09-12T07:19:42.451807+0000","last_became_peered":"2024-09-12T07:19:42.451807+0000","last_unstale":"2024-09-12T07:20:02.446281+0000","last_undegraded":"2024-09-12T07:20:02.446281+0000","last_fullsized":"2024-09-12T07:20:02.446281+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:24.806610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048363399999999998,"stat_sum":{"num_bytes":3282746,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":1137,"num_read_kb":973,"num_write":110,"num_write_kb":258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.224110+0000","last_change":"2024-09-12T07:19:53.499895+0000","last_active":"2024-09-12T07:20:02.224110+0000","last_peered":"2024-09-12T07:20:02.224110+0000","last_clean":"2024-09-12T07:20:02.224110+0000","last_became_active":"2024-09-12T07:19:53.499418+0000","last_became_peered":"2024-09-12T07:19:53.499418+0000","last_unstale":"2024-09-12T07:20:02.224110+0000","last_undegraded":"2024-09-12T07:20:02.224110+0000","last_fullsized":"2024-09-12T07:20:02.224110+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:21.122316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"41'53","reported_seq":503,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.459821+0000","last_change":"2024-09-12T07:19:44.455560+0000","last_active":"2024-09-12T07:20:02.459821+0000","last_peered":"2024-09-12T07:20:02.459821+0000","last_clean":"2024-09-12T07:20:02.459821+0000","last_became_active":"2024-09-12T07:19:42.451970+0000","last_became_peered":"2024-09-12T07:19:42.451970+0000","last_unstale":"2024-09-12T07:20:02.459821+0000","last_undegraded":"2024-09-12T07:20:02.459821+0000","last_fullsized":"2024-09-12T07:20:02.459821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:11:23.694124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083352100000000004,"stat_sum":{"num_bytes":3196961,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":1166,"num_read_kb":1044,"num_write":58,"num_write_kb":2081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":60,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.149407+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:20:02.149407+0000","last_peered":"2024-09-12T07:20:02.149407+0000","last_clean":"2024-09-12T07:20:02.149407+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:20:02.149407+0000","last_undegraded":"2024-09-12T07:20:02.149407+0000","last_fullsized":"2024-09-12T07:20:02.149407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"42'55","reported_seq":146,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.157978+0000","last_change":"2024-09-12T07:19:53.538615+0000","last_active":"2024-09-12T07:20:02.157978+0000","last_peered":"2024-09-12T07:20:02.157978+0000","last_clean":"2024-09-12T07:20:02.157978+0000","last_became_active":"2024-09-12T07:19:53.502164+0000","last_became_peered":"2024-09-12T07:19:53.502164+0000","last_unstale":"2024-09-12T07:20:02.157978+0000","last_undegraded":"2024-09-12T07:20:02.157978+0000","last_fullsized":"2024-09-12T07:20:02.157978+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:13:24.314004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089638599999999999,"stat_sum":{"num_bytes":3142058,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":177,"num_read_kb":227,"num_write":78,"num_write_kb":467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":100542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":61,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.153028+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:20:02.153028+0000","last_peered":"2024-09-12T07:20:02.153028+0000","last_clean":"2024-09-12T07:20:02.153028+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:20:02.153028+0000","last_undegraded":"2024-09-12T07:20:02.153028+0000","last_fullsized":"2024-09-12T07:20:02.153028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":151,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.149518+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:20:02.149518+0000","last_peered":"2024-09-12T07:20:02.149518+0000","last_clean":"2024-09-12T07:20:02.149518+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:20:02.149518+0000","last_undegraded":"2024-09-12T07:20:02.149518+0000","last_fullsized":"2024-09-12T07:20:02.149518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"41'31","reported_seq":131,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.149190+0000","last_change":"2024-09-12T07:19:53.521367+0000","last_active":"2024-09-12T07:20:02.149190+0000","last_peered":"2024-09-12T07:20:02.149190+0000","last_clean":"2024-09-12T07:20:02.149190+0000","last_became_active":"2024-09-12T07:19:53.501094+0000","last_became_peered":"2024-09-12T07:19:53.501094+0000","last_unstale":"2024-09-12T07:20:02.149190+0000","last_undegraded":"2024-09-12T07:20:02.149190+0000","last_fullsized":"2024-09-12T07:20:02.149190+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:47.486124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047491800000000001,"stat_sum":{"num_bytes":3860875,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":101,"num_read_kb":385,"num_write":40,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":128643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"39'84","reported_seq":653,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.525562+0000","last_change":"2024-09-12T07:19:44.469508+0000","last_active":"2024-09-12T07:20:02.525562+0000","last_peered":"2024-09-12T07:20:02.525562+0000","last_clean":"2024-09-12T07:20:02.525562+0000","last_became_active":"2024-09-12T07:19:42.450230+0000","last_became_peered":"2024-09-12T07:19:42.450230+0000","last_unstale":"2024-09-12T07:20:02.525562+0000","last_undegraded":"2024-09-12T07:20:02.525562+0000","last_fullsized":"2024-09-12T07:20:02.525562+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:04.100968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076460899999999997,"stat_sum":{"num_bytes":20347489,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":1387,"num_read_kb":2442,"num_write":79,"num_write_kb":1217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"42'18","reported_seq":76,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.158496+0000","last_change":"2024-09-12T07:19:44.480981+0000","last_active":"2024-09-12T07:20:02.158496+0000","last_peered":"2024-09-12T07:20:02.158496+0000","last_clean":"2024-09-12T07:20:02.158496+0000","last_became_active":"2024-09-12T07:19:42.452637+0000","last_became_peered":"2024-09-12T07:19:42.452637+0000","last_unstale":"2024-09-12T07:20:02.158496+0000","last_undegraded":"2024-09-12T07:20:02.158496+0000","last_fullsized":"2024-09-12T07:20:02.158496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:25.797127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000680892,"stat_sum":{"num_bytes":884736,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":11,"num_read_kb":8,"num_write":17,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"41'17","reported_seq":152,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.480650+0000","last_change":"2024-09-12T07:19:53.527716+0000","last_active":"2024-09-12T07:20:02.480650+0000","last_peered":"2024-09-12T07:20:02.480650+0000","last_clean":"2024-09-12T07:20:02.480650+0000","last_became_active":"2024-09-12T07:19:53.500926+0000","last_became_peered":"2024-09-12T07:19:53.500926+0000","last_unstale":"2024-09-12T07:20:02.480650+0000","last_undegraded":"2024-09-12T07:20:02.480650+0000","last_fullsized":"2024-09-12T07:20:02.480650+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:14:50.604581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054644100000000005,"stat_sum":{"num_bytes":2514028,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":102,"num_read_kb":549,"num_write":15,"num_write_kb":1064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":532572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"41'16","reported_seq":159,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.518064+0000","last_change":"2024-09-12T07:19:53.545055+0000","last_active":"2024-09-12T07:20:02.518064+0000","last_peered":"2024-09-12T07:20:02.518064+0000","last_clean":"2024-09-12T07:20:02.518064+0000","last_became_active":"2024-09-12T07:19:53.502246+0000","last_became_peered":"2024-09-12T07:19:53.502246+0000","last_unstale":"2024-09-12T07:20:02.518064+0000","last_undegraded":"2024-09-12T07:20:02.518064+0000","last_fullsized":"2024-09-12T07:20:02.518064+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:06.543538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050446300000000003,"stat_sum":{"num_bytes":9884978,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":83,"num_read_kb":651,"num_write":15,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":60385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"42'34","reported_seq":193,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.378730+0000","last_change":"2024-09-12T07:19:44.482470+0000","last_active":"2024-09-12T07:20:02.378730+0000","last_peered":"2024-09-12T07:20:02.378730+0000","last_clean":"2024-09-12T07:20:02.378730+0000","last_became_active":"2024-09-12T07:19:42.450577+0000","last_became_peered":"2024-09-12T07:19:42.450577+0000","last_unstale":"2024-09-12T07:20:02.378730+0000","last_undegraded":"2024-09-12T07:20:02.378730+0000","last_fullsized":"2024-09-12T07:20:02.378730+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:34.810513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317836,"stat_sum":{"num_bytes":3325952,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":287,"num_read_kb":1244,"num_write":31,"num_write_kb":1097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"41'91","reported_seq":616,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.473726+0000","last_change":"2024-09-12T07:19:44.481850+0000","last_active":"2024-09-12T07:20:02.473726+0000","last_peered":"2024-09-12T07:20:02.473726+0000","last_clean":"2024-09-12T07:20:02.473726+0000","last_became_active":"2024-09-12T07:19:42.452236+0000","last_became_peered":"2024-09-12T07:19:42.452236+0000","last_unstale":"2024-09-12T07:20:02.473726+0000","last_undegraded":"2024-09-12T07:20:02.473726+0000","last_fullsized":"2024-09-12T07:20:02.473726+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:43:51.412978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012939710000000001,"stat_sum":{"num_bytes":1260844,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":1409,"num_read_kb":1303,"num_write":83,"num_write_kb":659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"42'164","reported_seq":388,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.453314+0000","last_change":"2024-09-12T07:19:44.482179+0000","last_active":"2024-09-12T07:20:02.453314+0000","last_peered":"2024-09-12T07:20:02.453314+0000","last_clean":"2024-09-12T07:20:02.453314+0000","last_became_active":"2024-09-12T07:19:42.452155+0000","last_became_peered":"2024-09-12T07:19:42.452155+0000","last_unstale":"2024-09-12T07:20:02.453314+0000","last_undegraded":"2024-09-12T07:20:02.453314+0000","last_fullsized":"2024-09-12T07:20:02.453314+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:51.339782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011884759999999999,"stat_sum":{"num_bytes":957551,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":1165,"num_read_kb":948,"num_write":230,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"41'6","reported_seq":64,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.158380+0000","last_change":"2024-09-12T07:19:44.481020+0000","last_active":"2024-09-12T07:20:02.158380+0000","last_peered":"2024-09-12T07:20:02.158380+0000","last_clean":"2024-09-12T07:20:02.158380+0000","last_became_active":"2024-09-12T07:19:42.450535+0000","last_became_peered":"2024-09-12T07:19:42.450535+0000","last_unstale":"2024-09-12T07:20:02.158380+0000","last_undegraded":"2024-09-12T07:20:02.158380+0000","last_fullsized":"2024-09-12T07:20:02.158380+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:53:45.505347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073032100000000003,"stat_sum":{"num_bytes":1048576,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"41'36","reported_seq":186,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.535579+0000","last_change":"2024-09-12T07:19:44.481275+0000","last_active":"2024-09-12T07:20:02.535579+0000","last_peered":"2024-09-12T07:20:02.535579+0000","last_clean":"2024-09-12T07:20:02.535579+0000","last_became_active":"2024-09-12T07:19:42.452104+0000","last_became_peered":"2024-09-12T07:19:42.452104+0000","last_unstale":"2024-09-12T07:20:02.535579+0000","last_undegraded":"2024-09-12T07:20:02.535579+0000","last_fullsized":"2024-09-12T07:20:02.535579+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:25.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079083699999999996,"stat_sum":{"num_bytes":11510788,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":109,"num_read_kb":9964,"num_write":35,"num_write_kb":7165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"40'8","reported_seq":84,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.201353+0000","last_change":"2024-09-12T07:19:44.456086+0000","last_active":"2024-09-12T07:20:02.201353+0000","last_peered":"2024-09-12T07:20:02.201353+0000","last_clean":"2024-09-12T07:20:02.201353+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:20:02.201353+0000","last_undegraded":"2024-09-12T07:20:02.201353+0000","last_fullsized":"2024-09-12T07:20:02.201353+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:01:49.831173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060746,"stat_sum":{"num_bytes":6829634,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":20,"num_read_kb":1895,"num_write":13,"num_write_kb":1694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"40'61","reported_seq":12683,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T07:20:02.538545+0000","last_change":"2024-09-12T07:19:44.482564+0000","last_active":"2024-09-12T07:20:02.538545+0000","last_peered":"2024-09-12T07:20:02.538545+0000","last_clean":"2024-09-12T07:20:02.538545+0000","last_became_active":"2024-09-12T07:19:42.456127+0000","last_became_peered":"2024-09-12T07:19:42.456127+0000","last_unstale":"2024-09-12T07:20:02.538545+0000","last_undegraded":"2024-09-12T07:20:02.538545+0000","last_fullsized":"2024-09-12T07:20:02.538545+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:30:22.533631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793039999999999,"stat_sum":{"num_bytes":17591797,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":12606,"num_read_kb":23136,"num_write":58,"num_write_kb":7976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":225176379,"num_objects":210,"num_object_clones":33,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":60,"num_read":68609,"num_read_kb":114247,"num_write":1843,"num_write_kb":46920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1124955,"num_keys_recovered":12,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":78729216,"data_stored":84842066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1551,"ondisk_log_size":1551,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542155,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60056,"kb_used_data":28868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311784,"statfs":{"total":96636764160,"available":96575266816,"internally_reserved":0,"allocated":29560832,"data_stored":29676342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28736,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343104,"statfs":{"total":96636764160,"available":96607338496,"internally_reserved":0,"allocated":1744896,"data_stored":1466504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":5812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":5951488,"data_stored":8238770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61876,"kb_used_data":30688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309964,"statfs":{"total":96636764160,"available":96573403136,"internally_reserved":0,"allocated":31424512,"data_stored":33034797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":60129542155,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542155,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34136,"kb_used_data":2944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337704,"statfs":{"total":96636764160,"available":96601808896,"internally_reserved":0,"allocated":3014656,"data_stored":2706153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37244,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334596,"statfs":{"total":96636764160,"available":96598626304,"internally_reserved":0,"allocated":1937408,"data_stored":1559130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43464,"kb_used_data":8100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328376,"statfs":{"total":96636764160,"available":96592257024,"internally_reserved":0,"allocated":8294400,"data_stored":9022495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7241728,"data_stored":8240643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1478656,"data_stored":1367703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2555904,"data_stored":2514669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1232896,"data_stored":1677356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":30953472,"data_stored":32837162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5488640,"data_stored":8047267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1273856,"data_stored":1268869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28504064,"data_stored":28888397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:20:04.962 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-12T07:20:04.962 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:20:04.962 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-12T07:20:04.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-12T07:20:04.964 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:04.964 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:04.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:04.965 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:04.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.965 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:04.965 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:04.965 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.416519817, 2024-09-12T07:20:04.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:20:04.965 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.965 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.965 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.965 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:20:04.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:20:04.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:20:04.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:20:04.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:20:04.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:20:04.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:20:04.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:20:04.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.41649928, 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.970 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.971 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:20:04.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.971 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:04.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:04.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:04.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:20:04.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:20:04.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:20:04.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:20:04.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.4164792770000001, 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:20:04.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:20:04.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.416459328, 2024-09-12T07:20:04.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:04.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:20:04.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.41643846, 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.984 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:20:04.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.988 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.4164167590000001, 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.989 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:20:04.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:20:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.416396319, 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.993 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:20:04.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:04.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.416368598, 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:20:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:04.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:20:05.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:05.002 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.416345663, 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:05.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:20:05.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.4163253629999999, 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:05.011 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.416305087, 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:05.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:05.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:05.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:05.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:05.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:20:05.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:20:05.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.4162849280000001, 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:05.016 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:05.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:05.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:20:05.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:20:05.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:20:05.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:20:05.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.41614773, 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:05.022 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:05.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:20:05.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.026 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.416125082, 2024-09-12T07:20:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:20:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:20:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.027 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:20:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68088 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 393216~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.409682323, 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10346986399999999, 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:05.031 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68088 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555923+0000", 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.891e-06 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555929+0000", 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0549999999999996e-06 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555934+0000", 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6419999999999998e-06 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659060+0000", 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103125862 2024-09-12T07:20:05.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659139+0000", 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9450000000000007e-05 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659391+0000", 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025158400000000001 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.4096630960000001, 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:20:05.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.4094334589999999, 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:05.040 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.044 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.402509768, 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:20:05.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:20:05.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.402150185, 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:05.049 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:05.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:20:05.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.402070199, 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.054 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:05.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:05.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:05.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:05.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:20:05.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:20:05.058 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.058 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.058 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.058 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:05.058 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:05.058 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:05.059 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:05.060 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:05.061 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:05.061+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:05.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:05.062 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:05.062 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:05.062 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:05.063 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:05.063 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:05.063 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:05.063 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:05.066 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:05.161 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:05.162+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:20:05.179 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:05.179 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:05.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:05.261 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:05.263+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:05.282 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone9 from 0xc4227d3 (overlap 0x9a678f) to 0x6c154a 2024-09-12T07:20:05.362 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:05.363+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:05.365 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:05.365 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:05.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:05.417 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-12T07:20:05.418+0000 7f0757424640 -1 mon.a@0(leader).osd e45 definitely_dead 0 2024-09-12T07:20:05.424 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-12T07:20:05.425 INFO:teuthology.orchestra.run.smithi046.stdout:1056 write 0x6a483c7 thru 0x6a50817 (0x8451 bytes) 2024-09-12T07:20:05.433 INFO:teuthology.orchestra.run.smithi046.stdout:1057 write 0x4eb25a2 thru 0x4ebe046 (0xbaa5 bytes) 2024-09-12T07:20:05.451 INFO:teuthology.orchestra.run.smithi046.stdout:1058 trunc from 0xc4227d3 to 0x1cbba9 2024-09-12T07:20:05.462 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:05.464+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:05.506 INFO:teuthology.orchestra.run.smithi046.stdout:1059 read 0xfa7cf thru 0x106da5 (0xc5d7 bytes) 2024-09-12T07:20:05.514 INFO:teuthology.orchestra.run.smithi046.stdout:1062 writesame 0x1d95efb thru 0x1d96a25 data_size 0x1(0xb2b bytes) 2024-09-12T07:20:05.515 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.525 INFO:teuthology.orchestra.run.smithi046.stdout:1067 write 0x6644718 thru 0x66515e1 (0xceca bytes) 2024-09-12T07:20:05.529 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.529 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.543 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:05.543 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:05.543 INFO:teuthology.orchestra.run.smithi046.stdout:1070 write 0xe8fa93d thru 0xe8fef6e (0x4632 bytes) 2024-09-12T07:20:05.547 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.547 INFO:teuthology.orchestra.run.smithi046.stdout:1072 write 0x6a68900 thru 0x6a68c29 (0x32a bytes) 2024-09-12T07:20:05.549 INFO:teuthology.orchestra.run.smithi046.stdout:1075 punch from 0x5e9d71a to 0x5ea172c, (0x4012 bytes) 2024-09-12T07:20:05.549 INFO:teuthology.orchestra.run.smithi046.stdout:1076 write 0x3b2b9aa thru 0x3b34d11 (0x9368 bytes) 2024-09-12T07:20:05.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:05.556 INFO:teuthology.orchestra.run.smithi046.stdout:1079 trunc from 0xe8fef6f to 0xbf8419c 2024-09-12T07:20:05.563 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:05.564+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:05.566 INFO:teuthology.orchestra.run.smithi046.stdout:1080 punch from 0x3b285e to 0x3bbdc3, (0x9565 bytes) 2024-09-12T07:20:05.566 INFO:teuthology.orchestra.run.smithi046.stdout:1081 punch from 0x4332f00 to 0x433e678, (0xb778 bytes) 2024-09-12T07:20:05.566 INFO:teuthology.orchestra.run.smithi046.stdout:1082 trunc from 0xbf8419c to 0xc017759 2024-09-12T07:20:05.570 INFO:teuthology.orchestra.run.smithi046.stdout:1083 punch from 0xa1ae62a to 0xa1b7ff5, (0x99cb bytes) 2024-09-12T07:20:05.570 INFO:teuthology.orchestra.run.smithi046.stdout:1084 read 0x54f877 thru 0x55f3f9 (0xfb83 bytes) 2024-09-12T07:20:05.570 INFO:teuthology.orchestra.run.smithi046.stdout:1085 read 0x18317d thru 0x18bf4d (0x8dd1 bytes) 2024-09-12T07:20:05.571 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.571 INFO:teuthology.orchestra.run.smithi046.stdout:1087 write 0xa6a53dc thru 0xa6ab30c (0x5f31 bytes) 2024-09-12T07:20:05.573 INFO:teuthology.orchestra.run.smithi046.stdout:1088 writesame 0x4a3389e thru 0x4a4273e data_size 0x7(0xeea1 bytes) 2024-09-12T07:20:05.579 INFO:teuthology.orchestra.run.smithi046.stdout:1089 write 0xaed3e38 thru 0xaedf34e (0xb517 bytes) 2024-09-12T07:20:05.591 INFO:teuthology.orchestra.run.smithi046.stdout:1090 write 0xc5fb955 thru 0xc609828 (0xded4 bytes) 2024-09-12T07:20:05.597 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.597 INFO:teuthology.orchestra.run.smithi046.stdout:1093 punch from 0xb0d930 to 0xb19fb4, (0xc684 bytes) 2024-09-12T07:20:05.597 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.608 INFO:teuthology.orchestra.run.smithi046.stdout:1096 trunc from 0xc609829 to 0x2531abe 2024-09-12T07:20:05.620 INFO:teuthology.orchestra.run.smithi046.stdout:1097 punch from 0x202c5db to 0x202ddf8, (0x181d bytes) 2024-09-12T07:20:05.620 INFO:teuthology.orchestra.run.smithi046.stdout:1098 read 0xd18b4a thru 0xd26a35 (0xdeec bytes) 2024-09-12T07:20:05.620 INFO:teuthology.orchestra.run.smithi046.stdout:1101 read 0x1aa7f16 thru 0x1aadbf9 (0x5ce4 bytes) 2024-09-12T07:20:05.620 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.620 INFO:teuthology.orchestra.run.smithi046.stdout:1103 punch from 0x1b7a04d to 0x1b850dc, (0xb08f bytes) 2024-09-12T07:20:05.620 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.620 INFO:teuthology.orchestra.run.smithi046.stdout:1107 read 0xff2186 thru 0x100151d (0xf398 bytes) 2024-09-12T07:20:05.621 INFO:teuthology.orchestra.run.smithi046.stdout:1108 read 0x1000355 thru 0x1002a9b (0x2747 bytes) 2024-09-12T07:20:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:1109 write 0x5b84c9f thru 0x5b870f3 (0x2455 bytes) 2024-09-12T07:20:05.632 INFO:teuthology.orchestra.run.smithi046.stdout:1111 punch from 0x17e8f7c to 0x17eee4d, (0x5ed1 bytes) 2024-09-12T07:20:05.632 INFO:teuthology.orchestra.run.smithi046.stdout:1112 read 0x2ac3ccb thru 0x2aca70e (0x6a44 bytes) 2024-09-12T07:20:05.636 INFO:teuthology.orchestra.run.smithi046.stdout:1115 trunc from 0x5b870f4 to 0x29635f8 2024-09-12T07:20:05.645 INFO:teuthology.orchestra.run.smithi046.stdout:1120 punch from 0xdb8dbc to 0xdbd441, (0x4685 bytes) 2024-09-12T07:20:05.658 INFO:teuthology.orchestra.run.smithi046.stdout:1121 trunc from 0x29635f8 to 0xe2e8b08 2024-09-12T07:20:05.664 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:05.665+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:05.664 INFO:teuthology.orchestra.run.smithi046.stdout:1122 punch from 0x77d43e0 to 0x77dd986, (0x95a6 bytes) 2024-09-12T07:20:05.664 INFO:teuthology.orchestra.run.smithi046.stdout:1123 punch from 0xa647107 to 0xa64dc9d, (0x6b96 bytes) 2024-09-12T07:20:05.664 INFO:teuthology.orchestra.run.smithi046.stdout:1124 read 0x76f16c4 thru 0x76f3b37 (0x2474 bytes) 2024-09-12T07:20:05.665 INFO:teuthology.orchestra.run.smithi046.stdout:1126 writesame 0x5ee313 thru 0x5f2bb2 data_size 0x38(0x48a0 bytes) 2024-09-12T07:20:05.670 INFO:teuthology.orchestra.run.smithi046.stdout:1127 read 0xad1fc8e thru 0xad2af85 (0xb2f8 bytes) 2024-09-12T07:20:05.670 INFO:teuthology.orchestra.run.smithi046.stdout:1128 write 0xa111aed thru 0xa12197e (0xfe92 bytes) 2024-09-12T07:20:05.673 INFO:teuthology.orchestra.run.smithi046.stdout:1132 write 0x810c49f thru 0x810e161 (0x1cc3 bytes) 2024-09-12T07:20:05.676 INFO:teuthology.orchestra.run.smithi046.stdout:1133 read 0xda1be9c thru 0xda1efc2 (0x3127 bytes) 2024-09-12T07:20:05.676 INFO:teuthology.orchestra.run.smithi046.stdout:1134 writesame 0xbb86c36 thru 0xbb93fb0 data_size 0x1(0xd37b bytes) 2024-09-12T07:20:05.683 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:05.683 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:05.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:1135 read 0x3a49371 thru 0x3a4ace8 (0x1978 bytes) 2024-09-12T07:20:05.694 INFO:teuthology.orchestra.run.smithi046.stdout:1136 read 0xcc811fd thru 0xcc8af9e (0x9da2 bytes) 2024-09-12T07:20:05.708 INFO:teuthology.orchestra.run.smithi046.stdout:1137 trunc from 0xe2e8b08 to 0x2ca029 2024-09-12T07:20:05.721 INFO:teuthology.orchestra.run.smithi046.stdout:1139 read 0x1706c8 thru 0x17bd45 (0xb67e bytes) 2024-09-12T07:20:05.721 INFO:teuthology.orchestra.run.smithi046.stdout:1141 read 0x12c0c1 thru 0x12f190 (0x30d0 bytes) 2024-09-12T07:20:05.735 INFO:teuthology.orchestra.run.smithi046.stdout:1144 writesame 0x74b6c62 thru 0x74ba4e2 data_size 0xb(0x3881 bytes) 2024-09-12T07:20:05.735 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.736 INFO:teuthology.orchestra.run.smithi046.stdout:1146 trunc from 0x74ba4e3 to 0x8301fca 2024-09-12T07:20:05.741 INFO:teuthology.orchestra.run.smithi046.stdout:1147 write 0x69a12d5 thru 0x69ac97e (0xb6aa bytes) 2024-09-12T07:20:05.750 INFO:teuthology.orchestra.run.smithi046.stdout:1148 trunc from 0x8301fca to 0xcf3562e 2024-09-12T07:20:05.755 INFO:teuthology.orchestra.run.smithi046.stdout:1149 read 0x9acd582 thru 0x9ace908 (0x1387 bytes) 2024-09-12T07:20:05.755 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.756 INFO:teuthology.orchestra.run.smithi046.stdout:1151 write 0x54c8846 thru 0x54d4e1f (0xc5da bytes) 2024-09-12T07:20:05.759 INFO:teuthology.orchestra.run.smithi046.stdout:1153 punch from 0xc0ea3a3 to 0xc0ed5cd, (0x322a bytes) 2024-09-12T07:20:05.759 INFO:teuthology.orchestra.run.smithi046.stdout:1154 read 0x3c795b6 thru 0x3c84e0a (0xb855 bytes) 2024-09-12T07:20:05.759 INFO:teuthology.orchestra.run.smithi046.stdout:1155 write 0x90ce4f4 thru 0x90d97a4 (0xb2b1 bytes) 2024-09-12T07:20:05.763 INFO:teuthology.orchestra.run.smithi046.stdout:1157 punch from 0xe029d to 0xe2616, (0x2379 bytes) 2024-09-12T07:20:05.763 INFO:teuthology.orchestra.run.smithi046.stdout:1158 read 0x724440 thru 0x72de1d (0x99de bytes) 2024-09-12T07:20:05.764 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:05.765+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:05.770 INFO:teuthology.orchestra.run.smithi046.stdout:1160 trunc from 0xcf3562e to 0x685335e 2024-09-12T07:20:05.793 INFO:teuthology.orchestra.run.smithi046.stdout:1162 writesame 0xcf21f9f thru 0xcf27081 data_size 0x1(0x50e3 bytes) 2024-09-12T07:20:05.793 INFO:teuthology.orchestra.run.smithi046.stdout:1164 write 0x3829f39 thru 0x3839293 (0xf35b bytes) 2024-09-12T07:20:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:1166 read 0x877841e thru 0x878137f (0x8f62 bytes) 2024-09-12T07:20:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:1167 read 0xb403854 thru 0xb40be9a (0x8647 bytes) 2024-09-12T07:20:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:1168 write 0xd805e7c thru 0xd80b4f9 (0x567e bytes) 2024-09-12T07:20:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:1169 punch from 0x229ca34 to 0x22a1fe5, (0x55b1 bytes) 2024-09-12T07:20:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:1170 punch from 0x8d4b0e6 to 0x8d5751c, (0xc436 bytes) 2024-09-12T07:20:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:1171 read 0xccb9cc4 thru 0xccbc43d (0x277a bytes) 2024-09-12T07:20:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:1173 writesame 0xb64cec5 thru 0xb65009d data_size 0x1(0x31d9 bytes) 2024-09-12T07:20:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:1175 write 0xd1d045 thru 0xd1f04e (0x200a bytes) 2024-09-12T07:20:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:1177 write 0x3cc83d8 thru 0x3cd81f3 (0xfe1c bytes) 2024-09-12T07:20:05.825 INFO:teuthology.orchestra.run.smithi046.stdout:1178 punch from 0x42c1a85 to 0x42c38cb, (0x1e46 bytes) 2024-09-12T07:20:05.825 INFO:teuthology.orchestra.run.smithi046.stdout:1179 write 0x5b177a1 thru 0x5b1b672 (0x3ed2 bytes) 2024-09-12T07:20:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:1181 writesame 0x819b07d thru 0x819e7ad data_size 0x1(0x3731 bytes) 2024-09-12T07:20:05.831 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:05.831 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:05.834 INFO:teuthology.orchestra.run.smithi046.stdout:1184 writesame 0xa60fffe thru 0xa61c349 data_size 0x3a(0xc34c bytes) 2024-09-12T07:20:05.838 INFO:teuthology.orchestra.run.smithi046.stdout:1185 read 0x5ce8bbc thru 0x5ce98dc (0xd21 bytes) 2024-09-12T07:20:05.838 INFO:teuthology.orchestra.run.smithi046.stdout:1186 flatten 2024-09-12T07:20:05.841 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:05.845 INFO:teuthology.orchestra.run.smithi046.stdout:1188 write 0x720bbb5 thru 0x7219712 (0xdb5e bytes) 2024-09-12T07:20:05.849 INFO:teuthology.orchestra.run.smithi046.stdout:1189 read 0x199b791 thru 0x19a8357 (0xcbc7 bytes) 2024-09-12T07:20:05.850 INFO:teuthology.orchestra.run.smithi046.stdout:1190 read 0x2b9fdb7 thru 0x2bae36b (0xe5b5 bytes) 2024-09-12T07:20:05.850 INFO:teuthology.orchestra.run.smithi046.stdout:1191 punch from 0x98b6954 to 0x98bf8dc, (0x8f88 bytes) 2024-09-12T07:20:05.853 INFO:teuthology.orchestra.run.smithi046.stdout:1192 trunc from 0xd80b4fa to 0x9fe9554 2024-09-12T07:20:05.864 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:05.866+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:05.866 INFO:teuthology.orchestra.run.smithi046.stdout:1193 trunc from 0x9fe9554 to 0x736d679 2024-09-12T07:20:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:1194 trunc from 0x736d679 to 0x8e664a3 2024-09-12T07:20:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:1196 trunc from 0x8e664a3 to 0xe8b9ccd 2024-09-12T07:20:05.896 INFO:teuthology.orchestra.run.smithi046.stdout:1198 write 0xa9df62b thru 0xa9e0aba (0x1490 bytes) 2024-09-12T07:20:05.900 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:05.900 INFO:teuthology.orchestra.run.smithi046.stdout:1200 write 0xaf3b5be thru 0xaf41dec (0x682f bytes) 2024-09-12T07:20:05.903 INFO:teuthology.orchestra.run.smithi046.stdout:1203 write 0xd179ea2 thru 0xd188c2f (0xed8e bytes) 2024-09-12T07:20:05.907 INFO:teuthology.orchestra.run.smithi046.stdout:1205 write 0x6a72999 thru 0x6a7ef26 (0xc58e bytes) 2024-09-12T07:20:05.908 INFO:teuthology.orchestra.run.smithi046.stdout:1207 clone 11 order 18 su 1024 sc 3 2024-09-12T07:20:05.960 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:05.960 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:05.966 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:05.967+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:05.969 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:06.066 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:06.068+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:06.117 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:06.117 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:06.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:06.167 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:06.169+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:06.169 INFO:teuthology.orchestra.run.smithi046.stderr:marked in osd.3. 2024-09-12T07:20:06.188 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-12T07:20:06.268 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:06.270+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:06.281 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:06.281 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:06.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:06.369 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:06.371+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:06.397 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:06.397 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:06.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:06.470 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:06.472+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:06.514 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:06.514 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:06.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:06.571 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:06.572+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:06.629 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:06.630 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:06.638 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:06.671 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:06.673+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:06.772 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:06.774+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:06.786 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:06.786 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:06.794 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:06.873 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:06.875+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:06.907 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:06.907 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:06.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:06.975 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:06.976+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:07.022 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:07.022 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:07.030 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:07.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:07.114 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:07.116+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:07.177 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:07.178+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:07.183 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:07.183 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:07.184 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T07:20:07.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:07.228 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:07.228 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:07.255 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:07.255 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:07.276 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:07.278+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:07.288 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:07.315 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:07.315 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:07.335 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:07.345 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:07.345 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:07.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:07.356 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:07.377 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:07.378+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:07.389 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:07.389 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:07.422 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:07.422 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:07.478 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:07.480+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:07.506 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:07.506 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:07.506 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:07.506 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:07.506 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:07.515 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:07.579 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:07.581+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:07.624 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:07.624 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:07.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:07.624 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:07.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.624 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.624 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:07.624 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1431466760000006, 2024-09-12T07:20:07.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:20:07.624 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:20:07.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.628 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1431197960000006, 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.629 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:20:07.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:07.633 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1430957759999991, 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:07.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:07.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:20:07.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:20:07.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:20:07.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:20:07.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1430681299999996, 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.638 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:07.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.642 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.643 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:07.643 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1430446679999999, 2024-09-12T07:20:07.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:20:07.643 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.643 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.643 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.643 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.643 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.643 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.643 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:20:07.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.643 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:07.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1430221409999994, 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:20:07.647 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:20:07.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:20:07.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:20:07.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.651 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:20:07.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:20:07.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1430003380000002, 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:20:07.652 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.654 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:20:07.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:20:07.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:20:07.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:20:07.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:20:07.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1363665580000006, 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.657 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:20:07.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1363465500000007, 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.662 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:20:07.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1298635180000005, 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.666 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:20:07.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1298282309999994, 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.671 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:07.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:20:07.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1298055980000008, 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.676 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:07.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1297515429999994, 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:20:07.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:20:07.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:20:07.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:20:07.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:20:07.684 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.684 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.684 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1297326020000007, 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:20:07.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:20:07.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1297206650000007, 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:20:07.689 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:20:07.693 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1297090500000007, 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.694 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:20:07.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:20:07.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:20:07.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:20:07.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:20:07.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:20:07.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1226964420000005, 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.699 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:20:07.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1226727269999994, 2024-09-12T07:20:07.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:20:07.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.1225418699999992, 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:20:07.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:20:07.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:20:07.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:20:07.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:20:07.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.1588710799999999, 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:07.712 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:20:07.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:20:07.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:07.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:07.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:07.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:20:07.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:20:07.718 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.718 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.718 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.718 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:07.718 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:07.718 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:07.719 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:07.720 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:07.721+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:07.821 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:07.822+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:07.867 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:07.867 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:07.867 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:07.867 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:07.876 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:07.922 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:07.923+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:07.982 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:07.982 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:07.982 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:07.983 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:07.983 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:07.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:08.035 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:08.037+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:08.100 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:08.100 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:08.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:08.100 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:08.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.100 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:20:08.100 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:08.100 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 15.570740893, 2024-09-12T07:20:08.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:20:08.100 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.100 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:20:08.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:08.105 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7189432270000005, 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:08.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:08.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:20:08.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:20:08.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:20:08.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:20:08.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:20:08.110 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.110 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.110 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.110 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.110 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:08.110 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7188199500000003, 2024-09-12T07:20:08.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:20:08.111 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.111 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.111 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.111 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.111 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.111 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.111 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:20:08.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.112 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:08.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:08.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:20:08.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:20:08.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:20:08.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.116 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.116 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:08.116 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7186165590000009, 2024-09-12T07:20:08.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:20:08.116 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.116 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.116 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.117 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:20:08.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.117 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:08.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:08.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7184807319999997, 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.121 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:20:08.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7181540759999994, 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.125 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:20:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.130 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.130 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:08.130 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7181151880000005, 2024-09-12T07:20:08.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:20:08.130 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.130 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.130 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.130 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.130 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.130 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.130 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:20:08.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:20:08.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:20:08.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:20:08.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:20:08.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:20:08.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:20:08.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:20:08.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:20:08.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:20:08.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.134 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7177483139999996, 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:20:08.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.136 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:08.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:08.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:20:08.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:20:08.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:20:08.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:20:08.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7176134580000006, 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:20:08.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:20:08.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7172435670000006, 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.144 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:08.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.148 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7170533690000003, 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:20:08.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:20:08.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7168332900000003, 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7166896250000008, 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7164634529999994, 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:08.166 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7162630950000004, 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:08.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.170 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7158552119999992, 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.171 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:20:08.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7153893300000007, 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:20:08.175 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:20:08.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7137871199999992, 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:20:08.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.7132530849999998, 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:20:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.4570550149999999, 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:20:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:20:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.193 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:08.194 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is set 2024-09-12T07:20:08.195 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:08.195 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:08.196+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:08.196 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-12T07:20:08.217 INFO:teuthology.orchestra.run.smithi046.stdout:1208 punch from 0x766a9fc to 0x766f958, (0x4f5c bytes) 2024-09-12T07:20:08.221 INFO:teuthology.orchestra.run.smithi046.stdout:1209 read 0xc731963 thru 0xc734c92 (0x3330 bytes) 2024-09-12T07:20:08.222 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:08.222 INFO:teuthology.orchestra.run.smithi046.stdout:1211 write 0xe64cfcc thru 0xe65ca3b (0xfa70 bytes) 2024-09-12T07:20:08.233 INFO:teuthology.orchestra.run.smithi046.stdout:1212 write 0x3919e0d thru 0x3927c6f (0xde63 bytes) 2024-09-12T07:20:08.284 INFO:teuthology.orchestra.run.smithi046.stdout:1213 write 0x63e82fd thru 0x63f6316 (0xe01a bytes) 2024-09-12T07:20:08.289 INFO:teuthology.orchestra.run.smithi046.stdout:1214 punch from 0xd79d443 to 0xd7ad180, (0xfd3d bytes) 2024-09-12T07:20:08.289 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:08.289 INFO:teuthology.orchestra.run.smithi046.stdout:1216 read 0x5a62ffc thru 0x5a661ca (0x31cf bytes) 2024-09-12T07:20:08.290 INFO:teuthology.orchestra.run.smithi046.stdout:1217 read 0x7186d04 thru 0x718bf98 (0x5295 bytes) 2024-09-12T07:20:08.293 INFO:teuthology.orchestra.run.smithi046.stdout:1219 write 0x57362d1 thru 0x57371bb (0xeeb bytes) 2024-09-12T07:20:08.295 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:08.297+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:08.353 INFO:teuthology.orchestra.run.smithi046.stdout:1220 write 0x9837779 thru 0x983a9f2 (0x327a bytes) 2024-09-12T07:20:08.359 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:08.359 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:08.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.359 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:81627 3.16 e48/24 3:6a11dee9:::rbd_data.1160b2633c70.0000000000000260:head v 48'22, mlcod=45'20)", 2024-09-12T07:20:08.359 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:08.359442+0000", 2024-09-12T07:20:08.359 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0019620369999999998, 2024-09-12T07:20:08.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019830199999999998, 2024-09-12T07:20:08.359 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:08.359442+0000", 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:08.359442+0000", 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:08.359443+0000", 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1200000000000002e-07 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:08.359452+0000", 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1919999999999994e-06 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:08.359463+0000", 2024-09-12T07:20:08.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0647e-05 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:08.359468+0000", 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3630000000000002e-06 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:08.359525+0000", 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6749000000000002e-05 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:08.359545+0000", 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0199000000000001e-05 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:08.361424+0000", 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001878731 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 1 2024-09-12T07:20:08.363 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:1222 writesame 0x2026043 thru 0x2032836 data_size 0x86(0xc7f4 bytes) 2024-09-12T07:20:08.367 INFO:teuthology.orchestra.run.smithi046.stdout:1223 write 0xe4d2001 thru 0xe4d5023 (0x3023 bytes) 2024-09-12T07:20:08.368 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:08.373 INFO:teuthology.orchestra.run.smithi046.stdout:1224 punch from 0x1916d6 to 0x19e038, (0xc962 bytes) 2024-09-12T07:20:08.373 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:08.373 INFO:teuthology.orchestra.run.smithi046.stdout:1227 read 0xdc01aad thru 0xdc03e10 (0x2364 bytes) 2024-09-12T07:20:08.373 INFO:teuthology.orchestra.run.smithi046.stdout:1228 writesame 0x4ad0020 thru 0x4add373 data_size 0x5(0xd354 bytes) 2024-09-12T07:20:08.379 INFO:teuthology.orchestra.run.smithi046.stdout:1229 read 0x583b99f thru 0x583c673 (0xcd5 bytes) 2024-09-12T07:20:08.380 INFO:teuthology.orchestra.run.smithi046.stdout:1230 read 0x3b269b9 thru 0x3b29c28 (0x3270 bytes) 2024-09-12T07:20:08.387 INFO:teuthology.orchestra.run.smithi046.stdout:1231 trunc from 0xe8b9ccd to 0xa0197d0 2024-09-12T07:20:08.401 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:08.401 INFO:teuthology.orchestra.run.smithi046.stdout:1235 write 0x87ef45c thru 0x87f7b84 (0x8729 bytes) 2024-09-12T07:20:08.409 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:08.411+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:08.415 INFO:teuthology.orchestra.run.smithi046.stdout:1236 writesame 0xc735a55 thru 0xc73df16 data_size 0x1(0x84c2 bytes) 2024-09-12T07:20:08.415 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:08.415 INFO:teuthology.orchestra.run.smithi046.stdout:1238 write 0xc6b7103 thru 0xc6bdbed (0x6aeb bytes) 2024-09-12T07:20:08.420 INFO:teuthology.orchestra.run.smithi046.stdout:1239 punch from 0x7134a07 to 0x7141497, (0xca90 bytes) 2024-09-12T07:20:08.420 INFO:teuthology.orchestra.run.smithi046.stdout:1242 punch from 0x7317b32 to 0x73217ba, (0x9c88 bytes) 2024-09-12T07:20:08.420 INFO:teuthology.orchestra.run.smithi046.stdout:1243 writesame 0xc19dee0 thru 0xc1ad3a0 data_size 0x1(0xf4c1 bytes) 2024-09-12T07:20:08.437 INFO:teuthology.orchestra.run.smithi046.stdout:1244 punch from 0x8b7f83 to 0x8b9775, (0x17f2 bytes) 2024-09-12T07:20:08.437 INFO:teuthology.orchestra.run.smithi046.stdout:1245 writesame 0x5d422a9 thru 0x5d4324b data_size 0x1(0xfa3 bytes) 2024-09-12T07:20:08.442 INFO:teuthology.orchestra.run.smithi046.stdout:1246 read 0xa5e1ec3 thru 0xa5eebce (0xcd0c bytes) 2024-09-12T07:20:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:1247 trunc from 0xc73df17 to 0xacc46c 2024-09-12T07:20:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:1249 writesame 0x67ebdc5 thru 0x67ef1d3 data_size 0x1(0x340f bytes) 2024-09-12T07:20:08.488 INFO:teuthology.orchestra.run.smithi046.stdout:1250 read 0x154e7b3 thru 0x15551ef (0x6a3d bytes) 2024-09-12T07:20:08.494 INFO:teuthology.orchestra.run.smithi046.stdout:1252 trunc from 0x67ef1d4 to 0xc201dee 2024-09-12T07:20:08.496 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:08.498+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:1253 read 0x47505dc thru 0x47565e1 (0x6006 bytes) 2024-09-12T07:20:08.500 INFO:teuthology.orchestra.run.smithi046.stdout:1254 write 0x7af7b75 thru 0x7b072f7 (0xf783 bytes) 2024-09-12T07:20:08.508 INFO:teuthology.orchestra.run.smithi046.stdout:1255 read 0x719b44a thru 0x71a22a4 (0x6e5b bytes) 2024-09-12T07:20:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:1256 writesame 0xcc7b681 thru 0xcc83599 data_size 0x1(0x7f19 bytes) 2024-09-12T07:20:08.514 INFO:teuthology.orchestra.run.smithi046.stdout:1257 read 0xc185a90 thru 0xc186863 (0xdd4 bytes) 2024-09-12T07:20:08.516 INFO:teuthology.orchestra.run.smithi046.stdout:1258 trunc from 0xcc8359a to 0xaced86a 2024-09-12T07:20:08.522 INFO:teuthology.orchestra.run.smithi046.stdout:1259 writesame 0x2c2806d thru 0x2c2fd1f data_size 0x3(0x7cb3 bytes) 2024-09-12T07:20:08.523 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:08.523 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:08.523 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:08.524 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:08.524 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:1260 read 0x21a694e thru 0x21a8c4d (0x2300 bytes) 2024-09-12T07:20:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:1262 read 0x2ef1e73 thru 0x2ef9222 (0x73b0 bytes) 2024-09-12T07:20:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:1263 punch from 0x2c50d9f to 0x2c5fba0, (0xee01 bytes) 2024-09-12T07:20:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:1264 write 0x691183d thru 0x6917446 (0x5c0a bytes) 2024-09-12T07:20:08.534 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:1266 read 0x69a994a thru 0x69b7860 (0xdf17 bytes) 2024-09-12T07:20:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:1267 read 0x17e3cfa thru 0x17ee3f9 (0xa700 bytes) 2024-09-12T07:20:08.535 INFO:teuthology.orchestra.run.smithi046.stdout:1269 write 0x1d4463b thru 0x1d478b7 (0x327d bytes) 2024-09-12T07:20:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:1270 punch from 0x4695b55 to 0x46a5996, (0xfe41 bytes) 2024-09-12T07:20:08.538 INFO:teuthology.orchestra.run.smithi046.stdout:1271 write 0xa4b5d41 thru 0xa4b92ec (0x35ac bytes) 2024-09-12T07:20:08.542 INFO:teuthology.orchestra.run.smithi046.stdout:1272 write 0xfb573b thru 0xfb83ba (0x2c80 bytes) 2024-09-12T07:20:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:1274 write 0x5746cfe thru 0x5754450 (0xd753 bytes) 2024-09-12T07:20:08.554 INFO:teuthology.orchestra.run.smithi046.stdout:1275 writesame 0x338855d thru 0x338f600 data_size 0x1b(0x70a4 bytes) 2024-09-12T07:20:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:1276 read 0x245a16e thru 0x2464f99 (0xae2c bytes) 2024-09-12T07:20:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:1277 writesame 0x532e924 thru 0x533d2b9 data_size 0x1d(0xe996 bytes) 2024-09-12T07:20:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:1278 write 0x433f723 thru 0x434724a (0x7b28 bytes) 2024-09-12T07:20:08.571 INFO:teuthology.orchestra.run.smithi046.stdout:1279 write 0x534b248 thru 0x535aba5 (0xf95e bytes) 2024-09-12T07:20:08.581 INFO:teuthology.orchestra.run.smithi046.stdout:1280 punch from 0x5f3dacb to 0x5f424c8, (0x49fd bytes) 2024-09-12T07:20:08.581 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:08.581 INFO:teuthology.orchestra.run.smithi046.stdout:1282 read 0x6b6105b thru 0x6b62545 (0x14eb bytes) 2024-09-12T07:20:08.581 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:08.581 INFO:teuthology.orchestra.run.smithi046.stdout:1285 read 0x5d04e21 thru 0x5d0b801 (0x69e1 bytes) 2024-09-12T07:20:08.589 INFO:teuthology.orchestra.run.smithi046.stdout:1286 write 0xdbc7988 thru 0xdbcc010 (0x4689 bytes) 2024-09-12T07:20:08.597 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:08.598+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:08.605 INFO:teuthology.orchestra.run.smithi046.stdout:1287 trunc from 0xdbcc011 to 0x2ca8f31 2024-09-12T07:20:08.620 INFO:teuthology.orchestra.run.smithi046.stdout:1288 read 0xf7e45 thru 0xfafe9 (0x31a5 bytes) 2024-09-12T07:20:08.621 INFO:teuthology.orchestra.run.smithi046.stdout:1289 read 0x12215b6 thru 0x12278dd (0x6328 bytes) 2024-09-12T07:20:08.621 INFO:teuthology.orchestra.run.smithi046.stdout:1290 flatten 2024-09-12T07:20:08.659 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:08.659 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:08.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1108606390000002, 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1108401020000001, 2024-09-12T07:20:08.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:08.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:08.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:20:08.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1108200989999997, 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:20:08.669 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:20:08.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:20:08.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1108001500000002, 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.674 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:08.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1107792820000002, 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:20:08.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:20:08.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1107575809999997, 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.683 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:20:08.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1107371410000004, 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.687 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:20:08.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.692 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.692 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:08.692 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1107094200000001, 2024-09-12T07:20:08.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:20:08.692 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.692 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.692 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.692 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:20:08.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.696 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1106864850000004, 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:20:08.700 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1106661850000004, 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.701 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:20:08.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.705 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1106459089999996, 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:20:08.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1106257499999996, 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.710 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:20:08.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1104885519999996, 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:20:08.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1104659039999998, 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.719 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:20:08.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:20:08.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68088 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 393216~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1040231450000002, 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10346986399999999, 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68088 2024-09-12T07:20:08.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555923+0000", 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.891e-06 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555929+0000", 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0549999999999996e-06 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555934+0000", 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6419999999999998e-06 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659060+0000", 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103125862 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659139+0000", 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9450000000000007e-05 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659391+0000", 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025158400000000001 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1040039180000001, 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:20:08.728 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:20:08.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.732 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.1037742809999997, 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:20:08.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:20:08.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:20:08.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:20:08.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:20:08.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.738 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0968505899999998, 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:20:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:20:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.096491007, 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:20:08.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:08.747 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.0964110209999998, 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:08.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:08.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.752 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.752 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:08.752 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:08.752 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:08.753 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:08.754+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:08.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:08.854 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:08.855+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:08.906 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:08.906 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:08.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.906 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84555 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.907 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.857428+0000", 2024-09-12T07:20:08.907 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.036903681000000001, 2024-09-12T07:20:08.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036925540999999999, 2024-09-12T07:20:08.907 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84555 2024-09-12T07:20:08.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.857428+0000", 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.857428+0000", 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.857428+0000", 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.03e-07 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.857429+0000", 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.119e-06 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.857432+0000", 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.857435+0000", 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.587e-06 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84587 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863525+0000", 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030806317, 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030903282000000001, 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84587 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863525+0000", 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863525+0000", 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863526+0000", 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.28e-07 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863527+0000", 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.141e-06 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863530+0000", 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9689999999999999e-06 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863533+0000", 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.18e-06 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84619 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875002+0000", 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019329399000000001, 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019477759000000001, 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84619 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.914 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875002+0000", 2024-09-12T07:20:08.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875002+0000", 2024-09-12T07:20:08.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875003+0000", 2024-09-12T07:20:08.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.41e-07 2024-09-12T07:20:08.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875005+0000", 2024-09-12T07:20:08.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9120000000000001e-06 2024-09-12T07:20:08.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875008+0000", 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8979999999999998e-06 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875011+0000", 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9100000000000001e-06 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84651 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875491+0000", 2024-09-12T07:20:08.917 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018840761000000001, 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019038113999999998, 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84651 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875491+0000", 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875491+0000", 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875491+0000", 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2000000000000001e-07 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875492+0000", 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.173e-06 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875495+0000", 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1250000000000001e-06 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875498+0000", 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.666e-06 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.920 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84797 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881697+0000", 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012635026000000001, 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012858359, 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84797 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881697+0000", 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881697+0000", 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881697+0000", 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2200000000000004e-07 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881699+0000", 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3300000000000001e-06 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881708+0000", 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881713+0000", 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1429999999999997e-06 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84829 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888268+0000", 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0060641169999999999, 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063122639999999997, 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.924 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84829 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888268+0000", 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888268+0000", 2024-09-12T07:20:08.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888268+0000", 2024-09-12T07:20:08.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6699999999999999e-07 2024-09-12T07:20:08.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888270+0000", 2024-09-12T07:20:08.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9439999999999999e-06 2024-09-12T07:20:08.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888276+0000", 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9560000000000002e-06 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888280+0000", 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84861 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888974+0000", 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0053577219999999997, 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056289820000000003, 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84861 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888974+0000", 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888974+0000", 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888974+0000", 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3500000000000002e-07 2024-09-12T07:20:08.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888976+0000", 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0109999999999999e-06 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888983+0000", 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888987+0000", 2024-09-12T07:20:08.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7190000000000001e-06 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84556 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.857438+0000", 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.036893691999999999, 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037188602000000001, 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84556 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.857438+0000", 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.857438+0000", 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863029+0000", 2024-09-12T07:20:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055908399999999997 2024-09-12T07:20:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863033+0000", 2024-09-12T07:20:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4689999999999999e-06 2024-09-12T07:20:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863047+0000", 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3431000000000001e-05 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863056+0000", 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8519999999999993e-06 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84588 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.934 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863538+0000", 2024-09-12T07:20:08.935 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030793757000000001, 2024-09-12T07:20:08.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031112266, 2024-09-12T07:20:08.935 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.937 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84588 2024-09-12T07:20:08.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.937 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863538+0000", 2024-09-12T07:20:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863538+0000", 2024-09-12T07:20:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863538+0000", 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.46e-07 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863540+0000", 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8509999999999999e-06 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863546+0000", 2024-09-12T07:20:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.959e-06 2024-09-12T07:20:08.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863551+0000", 2024-09-12T07:20:08.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6659999999999999e-06 2024-09-12T07:20:08.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.940 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84620 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875015+0000", 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019316211999999999, 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019659629000000001, 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84620 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.941 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875015+0000", 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875015+0000", 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875016+0000", 2024-09-12T07:20:08.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6199999999999999e-07 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875017+0000", 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1820000000000001e-06 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875020+0000", 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1650000000000002e-06 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875023+0000", 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6410000000000002e-06 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84652 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875501+0000", 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018830517000000001, 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019198976, 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.944 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84652 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875501+0000", 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875501+0000", 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875501+0000", 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1199999999999999e-07 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875502+0000", 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.124e-06 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875506+0000", 2024-09-12T07:20:08.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0469999999999998e-06 2024-09-12T07:20:08.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875508+0000", 2024-09-12T07:20:08.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.666e-06 2024-09-12T07:20:08.947 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.948 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.949 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84798 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.949 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881719+0000", 2024-09-12T07:20:08.949 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012612668, 2024-09-12T07:20:08.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013027505999999999, 2024-09-12T07:20:08.949 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.949 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.949 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84798 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881719+0000", 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881719+0000", 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881719+0000", 2024-09-12T07:20:08.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6300000000000001e-07 2024-09-12T07:20:08.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881721+0000", 2024-09-12T07:20:08.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0090000000000002e-06 2024-09-12T07:20:08.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881728+0000", 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.224e-06 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881732+0000", 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7299999999999996e-06 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84830 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888286+0000", 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.006045946, 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065113619999999997, 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84830 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888286+0000", 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888286+0000", 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888286+0000", 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5100000000000001e-07 2024-09-12T07:20:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888288+0000", 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8530000000000001e-06 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888294+0000", 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8599999999999998e-06 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888298+0000", 2024-09-12T07:20:08.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6929999999999998e-06 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84862 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888994+0000", 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0053380759999999998, 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058520409999999997, 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84862 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888994+0000", 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888994+0000", 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888994+0000", 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888996+0000", 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1919999999999999e-06 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.889004+0000", 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.889009+0000", 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3600000000000004e-06 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84557 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863071+0000", 2024-09-12T07:20:08.959 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031260193999999998, 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031824428000000002, 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84557 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863071+0000", 2024-09-12T07:20:08.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863071+0000", 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863072+0000", 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0800000000000005e-07 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863074+0000", 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2139999999999999e-06 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863079+0000", 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2499999999999997e-06 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863083+0000", 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7299999999999999e-06 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84589 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863558+0000", 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030774097, 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031373926000000003, 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84589 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863558+0000", 2024-09-12T07:20:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863558+0000", 2024-09-12T07:20:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863558+0000", 2024-09-12T07:20:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.72e-07 2024-09-12T07:20:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863560+0000", 2024-09-12T07:20:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3149999999999999e-06 2024-09-12T07:20:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863566+0000", 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3539999999999999e-06 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863569+0000", 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0989999999999999e-06 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84621 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875026+0000", 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019305399000000001, 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019942673000000001, 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84621 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875026+0000", 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875026+0000", 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875027+0000", 2024-09-12T07:20:08.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.03e-07 2024-09-12T07:20:08.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875028+0000", 2024-09-12T07:20:08.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2860000000000001e-06 2024-09-12T07:20:08.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875031+0000", 2024-09-12T07:20:08.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.117e-06 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875034+0000", 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6299999999999998e-06 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84653 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875511+0000", 2024-09-12T07:20:08.970 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018820533, 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019487337, 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84653 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875511+0000", 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875511+0000", 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875511+0000", 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1e-07 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875512+0000", 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875515+0000", 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9059999999999998e-06 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875519+0000", 2024-09-12T07:20:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.224e-06 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84799 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881738+0000", 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012593666, 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013284038999999999, 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84799 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881738+0000", 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881738+0000", 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881738+0000", 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.41e-07 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881740+0000", 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9420000000000002e-06 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881746+0000", 2024-09-12T07:20:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8560000000000003e-06 2024-09-12T07:20:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881751+0000", 2024-09-12T07:20:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6e-06 2024-09-12T07:20:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84831 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888305+0000", 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0060268300000000004, 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0067402670000000003, 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84831 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888305+0000", 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888305+0000", 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888305+0000", 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5400000000000002e-07 2024-09-12T07:20:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888311+0000", 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6590000000000004e-06 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888317+0000", 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0750000000000003e-06 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888321+0000", 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5850000000000001e-06 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84863 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.889016+0000", 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0053157880000000001, 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060524949999999998, 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84863 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.889016+0000", 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.889016+0000", 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.889016+0000", 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9099999999999999e-07 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.889019+0000", 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3199999999999998e-06 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.889025+0000", 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.37e-06 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.889030+0000", 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5739999999999997e-06 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84558 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863088+0000", 2024-09-12T07:20:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031243426000000001, 2024-09-12T07:20:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032003730000000001, 2024-09-12T07:20:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84558 2024-09-12T07:20:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863088+0000", 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863088+0000", 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863089+0000", 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5199999999999998e-07 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863090+0000", 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.341e-06 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863093+0000", 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3179999999999998e-06 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863096+0000", 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6840000000000001e-06 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84590 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863573+0000", 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030758976, 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031564288000000003, 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84590 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863573+0000", 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863573+0000", 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.868975+0000", 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054022549999999999 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.868980+0000", 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8609999999999997e-06 2024-09-12T07:20:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.868991+0000", 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1338999999999999e-05 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.868999+0000", 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4769999999999997e-06 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84622 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875038+0000", 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019293250000000001, 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020147225000000001, 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84622 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875038+0000", 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875038+0000", 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875039+0000", 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3099999999999999e-07 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875043+0000", 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3869999999999997e-06 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875047+0000", 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5549999999999998e-06 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875049+0000", 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6599999999999999e-06 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84654 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875522+0000", 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018810072000000001, 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019713504999999999, 2024-09-12T07:20:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84654 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875522+0000", 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875522+0000", 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.880961+0000", 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054396829999999998 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.880967+0000", 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2850000000000002e-06 2024-09-12T07:20:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.880982+0000", 2024-09-12T07:20:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5446000000000001e-05 2024-09-12T07:20:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.880990+0000", 2024-09-12T07:20:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5019999999999992e-06 2024-09-12T07:20:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84800 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881756+0000", 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012575177, 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013526343, 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84800 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881756+0000", 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881756+0000", 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881757+0000", 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.41e-07 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881759+0000", 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.813e-06 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881765+0000", 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.037e-06 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881769+0000", 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6199999999999998e-06 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84832 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888338+0000", 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0059932470000000002, 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069826410000000004, 2024-09-12T07:20:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84832 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888338+0000", 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888338+0000", 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888339+0000", 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3799999999999997e-07 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888341+0000", 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3719999999999999e-06 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888349+0000", 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2150000000000004e-06 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888354+0000", 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2519999999999999e-06 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84559 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863099+0000", 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031232314000000001, 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032247953000000003, 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84559 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863099+0000", 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863099+0000", 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863100+0000", 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3099999999999999e-07 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863101+0000", 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.269e-06 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863104+0000", 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0240000000000002e-06 2024-09-12T07:20:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863106+0000", 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.553e-06 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84591 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869023+0000", 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.025308921000000002, 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026348690000000001, 2024-09-12T07:20:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84591 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869023+0000", 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869023+0000", 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869023+0000", 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.37e-07 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869026+0000", 2024-09-12T07:20:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7750000000000001e-06 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869034+0000", 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5750000000000003e-06 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869040+0000", 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.472e-06 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84623 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875054+0000", 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019277543000000001, 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020340687999999999, 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84623 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875054+0000", 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875054+0000", 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875054+0000", 2024-09-12T07:20:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2600000000000001e-07 2024-09-12T07:20:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875055+0000", 2024-09-12T07:20:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.124e-06 2024-09-12T07:20:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875059+0000", 2024-09-12T07:20:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0450000000000001e-06 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875061+0000", 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6649999999999999e-06 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84655 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881004+0000", 2024-09-12T07:20:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013327719999999999, 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014418558999999999, 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84655 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881004+0000", 2024-09-12T07:20:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881004+0000", 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881005+0000", 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0100000000000005e-07 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881007+0000", 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881015+0000", 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8660000000000006e-06 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881021+0000", 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.665e-06 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84801 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881776+0000", 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012556029, 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013670181999999999, 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84801 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881776+0000", 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881776+0000", 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881776+0000", 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.65e-07 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881778+0000", 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.875e-06 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881787+0000", 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8349999999999993e-06 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881792+0000", 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8770000000000001e-06 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84833 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888362+0000", 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0059697090000000001, 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0071086400000000003, 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84833 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888362+0000", 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888362+0000", 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888362+0000", 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888365+0000", 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.311e-06 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888371+0000", 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4659999999999996e-06 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888378+0000", 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6599999999999998e-06 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84560 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863110+0000", 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031222113999999999, 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032397383000000002, 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84560 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863110+0000", 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863110+0000", 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863110+0000", 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.17e-07 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863111+0000", 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.311e-06 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863114+0000", 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.039e-06 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863117+0000", 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6819999999999999e-06 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84592 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869048+0000", 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.025283783000000001, 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026549552000000001, 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84592 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869048+0000", 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869048+0000", 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869048+0000", 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9099999999999999e-07 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869050+0000", 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1730000000000002e-06 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869056+0000", 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9479999999999996e-06 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869062+0000", 2024-09-12T07:20:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1989999999999997e-06 2024-09-12T07:20:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84624 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875064+0000", 2024-09-12T07:20:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019267353000000001, 2024-09-12T07:20:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020587090999999998, 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84624 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875064+0000", 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875064+0000", 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875065+0000", 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1e-07 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875066+0000", 2024-09-12T07:20:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.113e-06 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875069+0000", 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9220000000000001e-06 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875071+0000", 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.565e-06 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84770 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881028+0000", 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013303498, 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014670743999999999, 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84770 2024-09-12T07:20:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881028+0000", 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881028+0000", 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881029+0000", 2024-09-12T07:20:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4700000000000002e-07 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881031+0000", 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2529999999999999e-06 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881040+0000", 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3519999999999996e-06 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881046+0000", 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1360000000000002e-06 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84802 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881797+0000", 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012534215, 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013929852, 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84802 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881797+0000", 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881797+0000", 2024-09-12T07:20:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881798+0000", 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6199999999999999e-07 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881800+0000", 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0109999999999999e-06 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881806+0000", 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2330000000000003e-06 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881810+0000", 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.335e-06 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84834 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888388+0000", 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0059437819999999999, 2024-09-12T07:20:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073645220000000001, 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84834 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888388+0000", 2024-09-12T07:20:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888388+0000", 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888388+0000", 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9499999999999998e-07 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888391+0000", 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2670000000000001e-06 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888397+0000", 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4710000000000004e-06 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888402+0000", 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2190000000000003e-06 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84561 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863120+0000", 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031211731999999999, 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032662189000000001, 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84561 2024-09-12T07:20:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863120+0000", 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863120+0000", 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863120+0000", 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0900000000000001e-07 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863121+0000", 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2389999999999999e-06 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863124+0000", 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0920000000000002e-06 2024-09-12T07:20:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863127+0000", 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6079999999999998e-06 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84593 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869068+0000", 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.025263598000000002, 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026737909000000001, 2024-09-12T07:20:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84593 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869068+0000", 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869068+0000", 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869069+0000", 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000003e-07 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869071+0000", 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.295e-06 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869077+0000", 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0580000000000003e-06 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869082+0000", 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.848e-06 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84625 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875074+0000", 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.01925754, 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020756995, 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84625 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875074+0000", 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875074+0000", 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875074+0000", 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3699999999999999e-07 2024-09-12T07:20:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875075+0000", 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2500000000000004e-07 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875078+0000", 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8499999999999998e-06 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875081+0000", 2024-09-12T07:20:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5170000000000002e-06 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84771 3.1f 3.b26a74ff (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881053+0000", 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013278697000000001, 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014801518, 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84771 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881053+0000", 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881053+0000", 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881053+0000", 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6699999999999999e-07 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881056+0000", 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2239999999999998e-06 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881064+0000", 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4940000000000003e-06 2024-09-12T07:20:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881069+0000", 2024-09-12T07:20:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1569999999999999e-06 2024-09-12T07:20:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84803 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881816+0000", 2024-09-12T07:20:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012515293, 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014078164000000001, 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84803 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881816+0000", 2024-09-12T07:20:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881816+0000", 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881817+0000", 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.58e-07 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881819+0000", 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8059999999999999e-06 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881825+0000", 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0970000000000002e-06 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881830+0000", 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7289999999999997e-06 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84835 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888409+0000", 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0059229779999999998, 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075375019999999997, 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84835 2024-09-12T07:20:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888409+0000", 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888409+0000", 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888409+0000", 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.15e-07 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888411+0000", 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0159999999999998e-06 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888417+0000", 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1199999999999999e-06 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888422+0000", 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8590000000000004e-06 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84562 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863130+0000", 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031201573, 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032873910999999999, 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84562 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863130+0000", 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863130+0000", 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863130+0000", 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0900000000000001e-07 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863137+0000", 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4910000000000002e-06 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863145+0000", 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8790000000000003e-06 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863150+0000", 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2530000000000004e-06 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84594 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869103+0000", 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.025229004999999999, 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026951267000000001, 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84594 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869103+0000", 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869103+0000", 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869103+0000", 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3099999999999998e-07 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869106+0000", 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.621e-06 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869121+0000", 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5201999999999999e-05 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869126+0000", 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0660000000000003e-06 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84626 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875083+0000", 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019248161999999999, 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021012207000000001, 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84626 2024-09-12T07:20:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875083+0000", 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875083+0000", 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875084+0000", 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2000000000000001e-07 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875085+0000", 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2100000000000001e-06 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875088+0000", 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9019999999999999e-06 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875090+0000", 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.542e-06 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84772 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881098+0000", 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013233396, 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015022110999999999, 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84772 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881098+0000", 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881098+0000", 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881099+0000", 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7100000000000002e-07 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881106+0000", 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1899999999999998e-06 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881115+0000", 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2499999999999995e-06 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881121+0000", 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1589999999999998e-06 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.080 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84804 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881836+0000", 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012495157999999999, 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014308705, 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84804 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881836+0000", 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881836+0000", 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881837+0000", 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4499999999999997e-07 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881839+0000", 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.294e-06 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881846+0000", 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3759999999999996e-06 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881852+0000", 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1179999999999997e-06 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.083 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84836 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888428+0000", 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0059032069999999997, 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0077399849999999996, 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.084 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84836 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888428+0000", 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888428+0000", 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888429+0000", 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4499999999999997e-07 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888431+0000", 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.508e-06 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888439+0000", 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2139999999999999e-06 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888444+0000", 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5709999999999997e-06 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84563 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863158+0000", 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031174075999999998, 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033039097000000003, 2024-09-12T07:20:09.087 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84563 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863158+0000", 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863158+0000", 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863158+0000", 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4200000000000001e-07 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863160+0000", 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.481e-06 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863166+0000", 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7080000000000002e-06 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863170+0000", 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3409999999999999e-06 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84595 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869137+0000", 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.025195090999999999, 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027084113, 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84595 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869137+0000", 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869137+0000", 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869137+0000", 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5499999999999998e-07 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869140+0000", 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5959999999999998e-06 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869153+0000", 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3137e-05 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869158+0000", 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0100000000000003e-06 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84627 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875093+0000", 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.01923826, 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021150947, 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84627 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875093+0000", 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875093+0000", 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875094+0000", 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.11e-07 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875095+0000", 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7000000000000003e-07 2024-09-12T07:20:09.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875097+0000", 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9170000000000002e-06 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875100+0000", 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.52e-06 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84773 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881130+0000", 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013201456, 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015149938999999999, 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84773 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.098 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881130+0000", 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881130+0000", 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881131+0000", 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7700000000000005e-07 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881133+0000", 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2220000000000001e-06 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881140+0000", 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1860000000000003e-06 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881145+0000", 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2109999999999997e-06 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84805 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881866+0000", 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012465188, 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014461129, 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.101 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84805 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881866+0000", 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881866+0000", 2024-09-12T07:20:09.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881867+0000", 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1500000000000005e-07 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881869+0000", 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3089999999999998e-06 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881878+0000", 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3280000000000006e-06 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881884+0000", 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.178e-06 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84837 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.104 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888450+0000", 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0058818489999999998, 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079298010000000002, 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84837 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888450+0000", 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888450+0000", 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888450+0000", 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.41e-07 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888452+0000", 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9149999999999999e-06 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888458+0000", 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8510000000000004e-06 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888463+0000", 2024-09-12T07:20:09.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6580000000000001e-06 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84564 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863174+0000", 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031157824000000001, 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033256784999999997, 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84564 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863174+0000", 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863174+0000", 2024-09-12T07:20:09.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863174+0000", 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8599999999999999e-07 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863175+0000", 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1999999999999999e-06 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863179+0000", 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1450000000000001e-06 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863182+0000", 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6180000000000002e-06 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84596 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.111 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869165+0000", 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.025166302000000002, 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027311634000000001, 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84596 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869165+0000", 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869165+0000", 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869166+0000", 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5199999999999998e-07 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869168+0000", 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3609999999999999e-06 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869174+0000", 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0739999999999998e-06 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869179+0000", 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4920000000000004e-06 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.114 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84628 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875103+0000", 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.01922871, 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021399444, 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84628 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875103+0000", 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875103+0000", 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875103+0000", 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2100000000000001e-07 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875104+0000", 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.009e-06 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875107+0000", 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9399999999999998e-06 2024-09-12T07:20:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875110+0000", 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5320000000000001e-06 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84774 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881153+0000", 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013178619000000001, 2024-09-12T07:20:09.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015374786999999999, 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84774 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881153+0000", 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881153+0000", 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881153+0000", 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.41e-07 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881155+0000", 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9769999999999999e-06 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881162+0000", 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881167+0000", 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8470000000000003e-06 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84806 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881891+0000", 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012440626, 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014661163, 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84806 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881891+0000", 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881891+0000", 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881891+0000", 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-07 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881893+0000", 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9709999999999998e-06 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881900+0000", 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1580000000000002e-06 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881904+0000", 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7790000000000003e-06 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84838 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888468+0000", 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0058636770000000003, 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081087369999999995, 2024-09-12T07:20:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84838 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888468+0000", 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888468+0000", 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888468+0000", 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3799999999999998e-07 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888470+0000", 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8160000000000001e-06 2024-09-12T07:20:09.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888477+0000", 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888481+0000", 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6709999999999998e-06 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84565 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863186+0000", 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031145932000000001, 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033417589999999997, 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84565 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.129 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863186+0000", 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863186+0000", 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863186+0000", 2024-09-12T07:20:09.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0900000000000001e-07 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863187+0000", 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1570000000000001e-06 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863190+0000", 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0309999999999999e-06 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863193+0000", 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5919999999999999e-06 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84597 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869185+0000", 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0251469, 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027445352999999999, 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.132 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84597 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869185+0000", 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869185+0000", 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869185+0000", 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0600000000000001e-07 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869187+0000", 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869192+0000", 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1719999999999998e-06 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869197+0000", 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2050000000000001e-06 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84629 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875114+0000", 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019217880999999999, 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021549663, 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84629 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875114+0000", 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875114+0000", 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875114+0000", 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3999999999999998e-07 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875115+0000", 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0699999999999999e-06 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875118+0000", 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0240000000000002e-06 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875121+0000", 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-12T07:20:09.138 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84775 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881173+0000", 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013158978, 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015542065000000001, 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84775 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881173+0000", 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881173+0000", 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881173+0000", 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2000000000000001e-07 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881175+0000", 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9180000000000001e-06 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881181+0000", 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9599999999999997e-06 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881186+0000", 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8380000000000001e-06 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84807 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881911+0000", 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012420745, 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014854146, 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84807 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881911+0000", 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881911+0000", 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881911+0000", 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.46e-07 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881917+0000", 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7710000000000003e-06 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881924+0000", 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6089999999999997e-06 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881929+0000", 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0699999999999997e-06 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84839 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.145 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888487+0000", 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0058449080000000002, 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00832659, 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84839 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888487+0000", 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888487+0000", 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888487+0000", 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888489+0000", 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9400000000000001e-06 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888495+0000", 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.896e-06 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888500+0000", 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6199999999999998e-06 2024-09-12T07:20:09.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84566 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863196+0000", 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031136066, 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033722641999999997, 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84566 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863196+0000", 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863196+0000", 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863196+0000", 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.29e-07 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863197+0000", 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5000000000000001e-07 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863200+0000", 2024-09-12T07:20:09.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9909999999999998e-06 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863202+0000", 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5119999999999998e-06 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84598 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869204+0000", 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.025127792999999999, 2024-09-12T07:20:09.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027744700000000001, 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84598 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869204+0000", 2024-09-12T07:20:09.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869204+0000", 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869204+0000", 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-07 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869211+0000", 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5679999999999996e-06 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869218+0000", 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7549999999999997e-06 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869223+0000", 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1109999999999999e-06 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.155 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84630 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875124+0000", 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019208001999999998, 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021852112, 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.156 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84630 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875124+0000", 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875124+0000", 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875124+0000", 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1e-07 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875125+0000", 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.043e-06 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875128+0000", 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.931e-06 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875130+0000", 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.616e-06 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84776 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881191+0000", 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.01314033, 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015808165999999998, 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84776 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881191+0000", 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881191+0000", 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881192+0000", 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3200000000000001e-07 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881193+0000", 2024-09-12T07:20:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7969999999999999e-06 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881199+0000", 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0039999999999996e-06 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881204+0000", 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.634e-06 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84808 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881968+0000", 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012363710999999999, 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015056346, 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84808 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.163 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881968+0000", 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881968+0000", 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881968+0000", 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881971+0000", 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3599999999999999e-06 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881979+0000", 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.841e-06 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881984+0000", 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4790000000000004e-06 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84840 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888505+0000", 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0058268720000000003, 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085474680000000008, 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84840 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888505+0000", 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888505+0000", 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888505+0000", 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.58e-07 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888507+0000", 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.956e-06 2024-09-12T07:20:09.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888513+0000", 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888518+0000", 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7639999999999996e-06 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84567 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863207+0000", 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031125106999999999, 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033898251999999997, 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84567 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863207+0000", 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863207+0000", 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863207+0000", 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.16e-07 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863208+0000", 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2300000000000001e-06 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863211+0000", 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863214+0000", 2024-09-12T07:20:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5770000000000001e-06 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84599 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869236+0000", 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.025095424000000002, 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027918861, 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84599 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869236+0000", 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869236+0000", 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869237+0000", 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000005e-07 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869239+0000", 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9069999999999999e-06 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869245+0000", 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0610000000000001e-06 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869249+0000", 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6190000000000002e-06 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84631 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875133+0000", 2024-09-12T07:20:09.176 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019198347000000001, 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022074928000000001, 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84631 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875133+0000", 2024-09-12T07:20:09.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875133+0000", 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875134+0000", 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2999999999999999e-07 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875135+0000", 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7399999999999991e-07 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875137+0000", 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9110000000000002e-06 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875140+0000", 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.61e-06 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.179 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84777 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881210+0000", 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013121717, 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016048504000000002, 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84777 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881210+0000", 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881210+0000", 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881210+0000", 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3799999999999998e-07 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881212+0000", 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.891e-06 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881218+0000", 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0070000000000003e-06 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881223+0000", 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7090000000000001e-06 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84809 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881991+0000", 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012340186, 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015296738000000001, 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84809 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881991+0000", 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881991+0000", 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881992+0000", 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6899999999999998e-07 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881994+0000", 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2369999999999999e-06 2024-09-12T07:20:09.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882001+0000", 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4350000000000002e-06 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882005+0000", 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84841 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888533+0000", 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0057986490000000003, 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008778625, 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84841 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888533+0000", 2024-09-12T07:20:09.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888533+0000", 2024-09-12T07:20:09.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888534+0000", 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1099999999999996e-07 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888536+0000", 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5339999999999998e-06 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888543+0000", 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3919999999999997e-06 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888549+0000", 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6019999999999999e-06 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84568 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863217+0000", 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031115080999999999, 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034123146, 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84568 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863217+0000", 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863217+0000", 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863217+0000", 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0800000000000001e-07 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863218+0000", 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1069999999999999e-06 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863221+0000", 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.915e-06 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863224+0000", 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2059999999999999e-06 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.193 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84600 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869255+0000", 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.025076662999999999, 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028109603, 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84600 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869255+0000", 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869255+0000", 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869255+0000", 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0800000000000001e-07 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869257+0000", 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8020000000000001e-06 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869264+0000", 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7109999999999998e-06 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869268+0000", 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2520000000000001e-06 2024-09-12T07:20:09.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84632 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875143+0000", 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019188651000000001, 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022246835, 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84632 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875143+0000", 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875143+0000", 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875143+0000", 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3699999999999999e-07 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875144+0000", 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1000000000000001e-06 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875147+0000", 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.847e-06 2024-09-12T07:20:09.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875150+0000", 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6539999999999999e-06 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84778 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881228+0000", 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013103208999999999, 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016185273999999999, 2024-09-12T07:20:09.200 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84778 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881228+0000", 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881228+0000", 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881229+0000", 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3799999999999998e-07 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881230+0000", 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.719e-06 2024-09-12T07:20:09.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881236+0000", 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9789999999999998e-06 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881241+0000", 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3819999999999997e-06 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.203 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84810 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.882012+0000", 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012319640999999999, 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015424917, 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84810 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882012+0000", 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882012+0000", 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882012+0000", 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4400000000000001e-07 2024-09-12T07:20:09.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882015+0000", 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1229999999999998e-06 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882021+0000", 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2380000000000002e-06 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882026+0000", 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8470000000000003e-06 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84842 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888556+0000", 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0057753240000000001, 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089233149999999994, 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84842 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888556+0000", 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888556+0000", 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888557+0000", 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8100000000000003e-07 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888559+0000", 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0810000000000001e-06 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888565+0000", 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2219999999999999e-06 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888570+0000", 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84569 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.210 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863227+0000", 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031104775000000001, 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034303252999999999, 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84569 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863227+0000", 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863227+0000", 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863227+0000", 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0900000000000001e-07 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863228+0000", 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.051e-06 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863231+0000", 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.948e-06 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863234+0000", 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.853e-06 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84601 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869274+0000", 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.025057717, 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028311006999999999, 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84601 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869274+0000", 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869274+0000", 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869274+0000", 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869276+0000", 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9290000000000001e-06 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869283+0000", 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2190000000000001e-06 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869287+0000", 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3150000000000002e-06 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84633 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875171+0000", 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019160667999999999, 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022469151, 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84633 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875171+0000", 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875171+0000", 2024-09-12T07:20:09.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875171+0000", 2024-09-12T07:20:09.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:09.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875174+0000", 2024-09-12T07:20:09.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.373e-06 2024-09-12T07:20:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875181+0000", 2024-09-12T07:20:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7499999999999997e-06 2024-09-12T07:20:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875186+0000", 2024-09-12T07:20:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.711e-06 2024-09-12T07:20:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.220 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84779 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881247+0000", 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013084451, 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016437139, 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.221 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84779 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881247+0000", 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881247+0000", 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881248+0000", 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0699999999999998e-07 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881250+0000", 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1449999999999998e-06 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881256+0000", 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5810000000000002e-06 2024-09-12T07:20:09.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881261+0000", 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9389999999999996e-06 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84811 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.882031+0000", 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012300291, 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015677587999999999, 2024-09-12T07:20:09.224 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84811 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882031+0000", 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882031+0000", 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882032+0000", 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3000000000000002e-07 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882034+0000", 2024-09-12T07:20:09.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.832e-06 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882039+0000", 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9490000000000001e-06 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882045+0000", 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7429999999999999e-06 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84843 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888577+0000", 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0057550020000000004, 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091577220000000001, 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84843 2024-09-12T07:20:09.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888577+0000", 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888577+0000", 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888577+0000", 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5999999999999999e-07 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888583+0000", 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888593+0000", 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0054999999999999e-05 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888599+0000", 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6149999999999996e-06 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84570 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863239+0000", 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031092957000000001, 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034522153999999999, 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.231 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84570 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863239+0000", 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863239+0000", 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863239+0000", 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863241+0000", 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.844e-06 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863249+0000", 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1499999999999999e-06 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863254+0000", 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2320000000000001e-06 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84602 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.234 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869293+0000", 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.025038465999999999, 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028493287999999999, 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84602 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869293+0000", 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869293+0000", 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869294+0000", 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5199999999999998e-07 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869295+0000", 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9479999999999998e-06 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869302+0000", 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1589999999999998e-06 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869306+0000", 2024-09-12T07:20:09.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6929999999999998e-06 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84634 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875194+0000", 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019138031, 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022616122999999998, 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.238 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84634 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875194+0000", 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875194+0000", 2024-09-12T07:20:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875194+0000", 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5699999999999998e-07 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875200+0000", 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9449999999999998e-06 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875206+0000", 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3169999999999998e-06 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875211+0000", 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2399999999999998e-06 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84780 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881287+0000", 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013044518999999999, 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016547227000000001, 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84780 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881287+0000", 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881287+0000", 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881288+0000", 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881290+0000", 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.587e-06 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881298+0000", 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7479999999999993e-06 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881304+0000", 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:20:09.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84812 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.882051+0000", 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012280617000000001, 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015806635999999999, 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84812 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882051+0000", 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882051+0000", 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882051+0000", 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2000000000000001e-07 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882053+0000", 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8679999999999999e-06 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882060+0000", 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7089999999999996e-06 2024-09-12T07:20:09.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882065+0000", 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8629999999999999e-06 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84844 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888608+0000", 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0057240540000000001, 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092920309999999992, 2024-09-12T07:20:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84844 2024-09-12T07:20:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888608+0000", 2024-09-12T07:20:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888608+0000", 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888608+0000", 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4000000000000002e-07 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888610+0000", 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2460000000000002e-06 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888619+0000", 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8829999999999995e-06 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888624+0000", 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2429999999999996e-06 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.251 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84571 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863276+0000", 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031055612999999999, 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034678707000000003, 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84571 2024-09-12T07:20:09.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863276+0000", 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863276+0000", 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863276+0000", 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.58e-07 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863279+0000", 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.119e-06 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863283+0000", 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8389999999999997e-06 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863287+0000", 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4790000000000001e-06 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84603 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869312+0000", 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.025019553999999999, 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.02869619, 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84603 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869312+0000", 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869312+0000", 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869312+0000", 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3200000000000001e-07 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869314+0000", 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7099999999999999e-06 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869320+0000", 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869333+0000", 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3509000000000001e-05 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84635 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875219+0000", 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019113075, 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022839435000000002, 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84635 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875219+0000", 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875219+0000", 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875219+0000", 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8700000000000001e-07 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875221+0000", 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1730000000000002e-06 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875227+0000", 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1759999999999998e-06 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875232+0000", 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9339999999999997e-06 2024-09-12T07:20:09.261 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84781 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881312+0000", 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013019563, 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016793226000000001, 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84781 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881312+0000", 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881312+0000", 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881312+0000", 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2899999999999999e-07 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881314+0000", 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9149999999999999e-06 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881321+0000", 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2260000000000002e-06 2024-09-12T07:20:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881326+0000", 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1410000000000004e-06 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84813 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.882071+0000", 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012260472, 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016065356999999999, 2024-09-12T07:20:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84813 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882071+0000", 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882071+0000", 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882072+0000", 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3200000000000001e-07 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882073+0000", 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.804e-06 2024-09-12T07:20:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882079+0000", 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.015e-06 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882084+0000", 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84845 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888642+0000", 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0056897900000000001, 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095189269999999999, 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84845 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888642+0000", 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888642+0000", 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888642+0000", 2024-09-12T07:20:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0800000000000004e-07 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888645+0000", 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7070000000000001e-06 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888653+0000", 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8299999999999996e-06 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888659+0000", 2024-09-12T07:20:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84572 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863292+0000", 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031040013000000002, 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034902580000000002, 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84572 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863292+0000", 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863292+0000", 2024-09-12T07:20:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863292+0000", 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.28e-07 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863293+0000", 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1880000000000001e-06 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863296+0000", 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.038e-06 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863299+0000", 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5210000000000001e-06 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84604 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869340+0000", 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024991675000000001, 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028878391999999999, 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84604 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869340+0000", 2024-09-12T07:20:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869340+0000", 2024-09-12T07:20:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869340+0000", 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869345+0000", 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.42e-06 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869351+0000", 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0739999999999998e-06 2024-09-12T07:20:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869356+0000", 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8470000000000003e-06 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84636 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875239+0000", 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019092839, 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023005076999999999, 2024-09-12T07:20:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84636 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875239+0000", 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875239+0000", 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875239+0000", 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.77e-07 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875242+0000", 2024-09-12T07:20:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.39e-06 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875248+0000", 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.161e-06 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875252+0000", 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.673e-06 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84782 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881332+0000", 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.013000104, 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016936861000000001, 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84782 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881332+0000", 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881332+0000", 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881332+0000", 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1399999999999998e-07 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881334+0000", 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7969999999999999e-06 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881340+0000", 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0700000000000003e-06 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881344+0000", 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7119999999999999e-06 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84814 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.882090+0000", 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012241577, 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016203713000000002, 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84814 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882090+0000", 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882090+0000", 2024-09-12T07:20:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882090+0000", 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.22e-07 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882092+0000", 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8390000000000001e-06 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882098+0000", 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9309999999999996e-06 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882103+0000", 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6870000000000002e-06 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84846 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888667+0000", 2024-09-12T07:20:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0056650909999999997, 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096550669999999998, 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84846 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888667+0000", 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888667+0000", 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888667+0000", 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3099999999999998e-07 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888669+0000", 2024-09-12T07:20:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.12e-06 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888676+0000", 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5200000000000003e-06 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888681+0000", 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.254e-06 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84573 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863302+0000", 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031029746, 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035071549, 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84573 2024-09-12T07:20:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863302+0000", 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863302+0000", 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863302+0000", 2024-09-12T07:20:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000002e-07 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863303+0000", 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.077e-06 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863309+0000", 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0260000000000004e-06 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863312+0000", 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7539999999999998e-06 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84605 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869362+0000", 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024969835999999999, 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029061297999999999, 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84605 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869362+0000", 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869362+0000", 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869362+0000", 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.84e-07 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869364+0000", 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8980000000000001e-06 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869370+0000", 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9070000000000004e-06 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869375+0000", 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5900000000000001e-06 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84637 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875258+0000", 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019073335, 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023220745000000001, 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84637 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875258+0000", 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875258+0000", 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875259+0000", 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7099999999999997e-07 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875261+0000", 2024-09-12T07:20:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.001e-06 2024-09-12T07:20:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875266+0000", 2024-09-12T07:20:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7429999999999999e-06 2024-09-12T07:20:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875271+0000", 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6800000000000001e-06 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84783 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881366+0000", 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012965905, 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017161869999999999, 2024-09-12T07:20:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84783 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881366+0000", 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881366+0000", 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881366+0000", 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2699999999999999e-07 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881368+0000", 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2139999999999999e-06 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881375+0000", 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9210000000000003e-06 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881381+0000", 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2220000000000001e-06 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84815 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.882108+0000", 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012223309999999999, 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016454429999999999, 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84815 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882108+0000", 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882108+0000", 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882109+0000", 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.41e-07 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882110+0000", 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7579999999999999e-06 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882117+0000", 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7209999999999997e-06 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882122+0000", 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5950000000000001e-06 2024-09-12T07:20:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84847 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888688+0000", 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0056441199999999999, 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098990629999999996, 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84847 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888688+0000", 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888688+0000", 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888688+0000", 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000005e-07 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888690+0000", 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1619999999999998e-06 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888696+0000", 2024-09-12T07:20:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1419999999999998e-06 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888701+0000", 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8890000000000001e-06 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84574 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863315+0000", 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031016517, 2024-09-12T07:20:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035296519999999998, 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84574 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863315+0000", 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863315+0000", 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863315+0000", 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1e-07 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863320+0000", 2024-09-12T07:20:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3640000000000001e-06 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863323+0000", 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1810000000000001e-06 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863326+0000", 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4479999999999999e-06 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84606 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869392+0000", 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024939722000000001, 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029245442999999999, 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84606 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869392+0000", 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869392+0000", 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869393+0000", 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6400000000000002e-07 2024-09-12T07:20:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869395+0000", 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2859999999999998e-06 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869401+0000", 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4579999999999998e-06 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869406+0000", 2024-09-12T07:20:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0760000000000002e-06 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84638 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875277+0000", 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019054694, 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023384815, 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84638 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875277+0000", 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875277+0000", 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875277+0000", 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6800000000000001e-07 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875279+0000", 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8729999999999999e-06 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875285+0000", 2024-09-12T07:20:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8830000000000003e-06 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875290+0000", 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7149999999999997e-06 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84784 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881387+0000", 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012944361, 2024-09-12T07:20:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017299360999999999, 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84784 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881387+0000", 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881387+0000", 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881388+0000", 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3599999999999999e-07 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881394+0000", 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881400+0000", 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3620000000000002e-06 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881406+0000", 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8660000000000002e-06 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84816 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.882128+0000", 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012203929000000001, 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016582208000000001, 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84816 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882128+0000", 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882128+0000", 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882128+0000", 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.84e-07 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882130+0000", 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0040000000000002e-06 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882136+0000", 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.981e-06 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882141+0000", 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8609999999999997e-06 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84848 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888707+0000", 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0056243329999999996, 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010027393000000001, 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84848 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888707+0000", 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888707+0000", 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888708+0000", 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888710+0000", 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.069e-06 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888719+0000", 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3209999999999994e-06 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888724+0000", 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2140000000000004e-06 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84575 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863330+0000", 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.031002018999999999, 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035448012000000001, 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84575 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863330+0000", 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863330+0000", 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863330+0000", 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2999999999999999e-07 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863331+0000", 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1170000000000001e-06 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863337+0000", 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0650000000000004e-06 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863344+0000", 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4629999999999998e-06 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84607 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869413+0000", 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024918928, 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.02941736, 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84607 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869413+0000", 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869413+0000", 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869413+0000", 2024-09-12T07:20:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5400000000000002e-07 2024-09-12T07:20:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869415+0000", 2024-09-12T07:20:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:20:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869422+0000", 2024-09-12T07:20:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2239999999999998e-06 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869427+0000", 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6310000000000002e-06 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84639 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875296+0000", 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019036132000000001, 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023584694, 2024-09-12T07:20:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84639 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875296+0000", 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875296+0000", 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875297+0000", 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6130000000000001e-06 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875299+0000", 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9589999999999998e-06 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875311+0000", 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1449e-05 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875316+0000", 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5979999999999996e-06 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84785 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881413+0000", 2024-09-12T07:20:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012918323000000001, 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017518783999999999, 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84785 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881413+0000", 2024-09-12T07:20:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881413+0000", 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881414+0000", 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6400000000000003e-07 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881416+0000", 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1849999999999998e-06 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881422+0000", 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5050000000000004e-06 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881427+0000", 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0150000000000003e-06 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84817 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.882155+0000", 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012176862, 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016825583000000002, 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84817 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882155+0000", 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882155+0000", 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882155+0000", 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000005e-07 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882158+0000", 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2979999999999999e-06 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882164+0000", 2024-09-12T07:20:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.754e-06 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882170+0000", 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2329999999999997e-06 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84849 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888731+0000", 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0056007929999999997, 2024-09-12T07:20:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010277250999999999, 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84849 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888731+0000", 2024-09-12T07:20:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888731+0000", 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888731+0000", 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.77e-07 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888733+0000", 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0310000000000001e-06 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888739+0000", 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8490000000000002e-06 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888744+0000", 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7849999999999999e-06 2024-09-12T07:20:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84576 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863356+0000", 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030975175000000001, 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035677185, 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84576 2024-09-12T07:20:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863356+0000", 2024-09-12T07:20:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863356+0000", 2024-09-12T07:20:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863357+0000", 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7700000000000005e-07 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863360+0000", 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863367+0000", 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3259999999999998e-06 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863372+0000", 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3590000000000001e-06 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84608 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869433+0000", 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024899042999999999, 2024-09-12T07:20:09.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029626436999999999, 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84608 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869433+0000", 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869433+0000", 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869433+0000", 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5899999999999996e-07 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869435+0000", 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.815e-06 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869447+0000", 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2514000000000001e-05 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869452+0000", 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6360000000000002e-06 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84640 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875324+0000", 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.019007988999999999, 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023759168000000001, 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84640 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875324+0000", 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875324+0000", 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875324+0000", 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2499999999999995e-07 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875326+0000", 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.295e-06 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875333+0000", 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0569999999999998e-06 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875337+0000", 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6999999999999999e-06 2024-09-12T07:20:09.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84786 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881435+0000", 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012896835000000001, 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017673400999999998, 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84786 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881435+0000", 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881435+0000", 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881435+0000", 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999999e-07 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881437+0000", 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1660000000000001e-06 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881444+0000", 2024-09-12T07:20:09.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5440000000000004e-06 2024-09-12T07:20:09.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881449+0000", 2024-09-12T07:20:09.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:20:09.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.368 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.368 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84818 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.882178+0000", 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012154023, 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016953909, 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84818 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882178+0000", 2024-09-12T07:20:09.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882178+0000", 2024-09-12T07:20:09.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882178+0000", 2024-09-12T07:20:09.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8599999999999998e-07 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882180+0000", 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882188+0000", 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1829999999999997e-06 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882193+0000", 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1150000000000002e-06 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84850 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888749+0000", 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0055824070000000002, 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010405526, 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84850 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888749+0000", 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888749+0000", 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888750+0000", 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4499999999999998e-07 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888751+0000", 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8780000000000001e-06 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888757+0000", 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7610000000000004e-06 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888762+0000", 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6210000000000003e-06 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84577 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.375 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863377+0000", 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030955044000000001, 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035821506000000003, 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84577 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863377+0000", 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863377+0000", 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863377+0000", 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-07 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863378+0000", 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.468e-06 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863382+0000", 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.275e-06 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863384+0000", 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6180000000000002e-06 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.378 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84609 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869458+0000", 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024873165999999999, 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029788528000000002, 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84609 2024-09-12T07:20:09.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869458+0000", 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869458+0000", 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869459+0000", 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8100000000000003e-07 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869462+0000", 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4820000000000001e-06 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869467+0000", 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6389999999999997e-06 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869472+0000", 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.775e-06 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84641 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875345+0000", 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018986785999999999, 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024083675999999998, 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.382 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84641 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875345+0000", 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875345+0000", 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875345+0000", 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2500000000000001e-07 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875347+0000", 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0889999999999998e-06 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875353+0000", 2024-09-12T07:20:09.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.677e-06 2024-09-12T07:20:09.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875358+0000", 2024-09-12T07:20:09.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7520000000000004e-06 2024-09-12T07:20:09.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.385 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84787 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881456+0000", 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012876122, 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017999294999999998, 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84787 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881456+0000", 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881456+0000", 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881456+0000", 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8799999999999998e-07 2024-09-12T07:20:09.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881458+0000", 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.934e-06 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881465+0000", 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.674e-06 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881470+0000", 2024-09-12T07:20:09.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1050000000000003e-06 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84819 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.882200+0000", 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012131680000000001, 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017280209000000001, 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84819 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882200+0000", 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.882200+0000", 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888009+0000", 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058086409999999998 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888017+0000", 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4819999999999994e-06 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888030+0000", 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2792000000000001e-05 2024-09-12T07:20:09.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888038+0000", 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7889999999999995e-06 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84851 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888767+0000", 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.005564355, 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010738836999999999, 2024-09-12T07:20:09.392 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84851 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888767+0000", 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888767+0000", 2024-09-12T07:20:09.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888768+0000", 2024-09-12T07:20:09.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.22e-07 2024-09-12T07:20:09.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888769+0000", 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.829e-06 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888775+0000", 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6169999999999998e-06 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888780+0000", 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6029999999999998e-06 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84578 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863387+0000", 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030944396999999998, 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036164415999999998, 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84578 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863387+0000", 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863387+0000", 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863387+0000", 2024-09-12T07:20:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2000000000000001e-07 2024-09-12T07:20:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863389+0000", 2024-09-12T07:20:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0640000000000001e-06 2024-09-12T07:20:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863392+0000", 2024-09-12T07:20:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9670000000000001e-06 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863394+0000", 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.503e-06 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84610 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869478+0000", 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024854016, 2024-09-12T07:20:09.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030100953999999999, 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84610 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869478+0000", 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869478+0000", 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869478+0000", 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1899999999999998e-07 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869484+0000", 2024-09-12T07:20:09.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9989999999999997e-06 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869490+0000", 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0000000000000002e-06 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869495+0000", 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0130000000000001e-06 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84642 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875390+0000", 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018941551000000001, 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024219350000000001, 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84642 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875390+0000", 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875390+0000", 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875390+0000", 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9799999999999999e-07 2024-09-12T07:20:09.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875392+0000", 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8610000000000001e-06 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875397+0000", 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7559999999999999e-06 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875400+0000", 2024-09-12T07:20:09.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3620000000000002e-06 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84788 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881476+0000", 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012855416, 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018179432999999998, 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84788 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881476+0000", 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881476+0000", 2024-09-12T07:20:09.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881477+0000", 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881479+0000", 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9939999999999999e-06 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881485+0000", 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.032e-06 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881489+0000", 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6210000000000003e-06 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84820 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.409 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888050+0000", 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0062813369999999997, 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011657875, 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84820 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888050+0000", 2024-09-12T07:20:09.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888050+0000", 2024-09-12T07:20:09.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888051+0000", 2024-09-12T07:20:09.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.99e-07 2024-09-12T07:20:09.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888054+0000", 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5909999999999998e-06 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888061+0000", 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3799999999999996e-06 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888068+0000", 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9609999999999999e-06 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84852 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888785+0000", 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0055464920000000001, 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010973388000000001, 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84852 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888785+0000", 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888785+0000", 2024-09-12T07:20:09.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888785+0000", 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1800000000000002e-07 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888787+0000", 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8619999999999999e-06 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888793+0000", 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9000000000000003e-06 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888798+0000", 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7480000000000001e-06 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.416 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84579 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863397+0000", 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030934555999999998, 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036415086999999999, 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84579 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863397+0000", 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863397+0000", 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863397+0000", 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1e-07 2024-09-12T07:20:09.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863398+0000", 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0690000000000001e-06 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863401+0000", 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0089999999999999e-06 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863404+0000", 2024-09-12T07:20:09.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7539999999999998e-06 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84611 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869517+0000", 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024814337999999998, 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030336117999999999, 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84611 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869517+0000", 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869517+0000", 2024-09-12T07:20:09.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869518+0000", 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2900000000000003e-07 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869520+0000", 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2589999999999999e-06 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869526+0000", 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0870000000000003e-06 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869531+0000", 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0699999999999997e-06 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84643 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875405+0000", 2024-09-12T07:20:09.423 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018926704999999999, 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024476003, 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84643 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875405+0000", 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875405+0000", 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875405+0000", 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3900000000000001e-07 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875406+0000", 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2130000000000001e-06 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875409+0000", 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0309999999999999e-06 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875412+0000", 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.897e-06 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.426 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84789 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881496+0000", 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012835352, 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018409058999999998, 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.427 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84789 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881496+0000", 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881496+0000", 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881497+0000", 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.27e-07 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881499+0000", 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0150000000000002e-06 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881505+0000", 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4450000000000001e-06 2024-09-12T07:20:09.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881510+0000", 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0710000000000002e-06 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84821 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888077+0000", 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0062548370000000001, 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011851682000000001, 2024-09-12T07:20:09.430 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84821 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888077+0000", 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888077+0000", 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888077+0000", 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.89e-07 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888080+0000", 2024-09-12T07:20:09.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2850000000000001e-06 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888086+0000", 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4389999999999997e-06 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888091+0000", 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2170000000000002e-06 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84853 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888804+0000", 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.005527723, 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011155986999999999, 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84853 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.434 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888804+0000", 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888804+0000", 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888804+0000", 2024-09-12T07:20:09.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7800000000000002e-07 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888806+0000", 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.035e-06 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888813+0000", 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1659999999999999e-06 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888818+0000", 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.057e-06 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84580 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863407+0000", 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030924224, 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036578909, 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84580 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863407+0000", 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863407+0000", 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863408+0000", 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.17e-07 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863409+0000", 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.051e-06 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863412+0000", 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0529999999999998e-06 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863414+0000", 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5459999999999998e-06 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84612 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869538+0000", 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024793729, 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030474107, 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84612 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869538+0000", 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869538+0000", 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869538+0000", 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2599999999999998e-07 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869547+0000", 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2670000000000006e-06 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869553+0000", 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3960000000000003e-06 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869558+0000", 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0629999999999996e-06 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.443 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84644 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875415+0000", 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018916181000000001, 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024619855, 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84644 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875415+0000", 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875415+0000", 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875416+0000", 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0899999999999997e-07 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875417+0000", 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.279e-06 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875420+0000", 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.01e-06 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875423+0000", 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4990000000000001e-06 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84790 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881516+0000", 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012815531, 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018563264999999999, 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84790 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881516+0000", 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881516+0000", 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881516+0000", 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.46e-07 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881518+0000", 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.663e-06 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881524+0000", 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881529+0000", 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8690000000000003e-06 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.450 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84822 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888097+0000", 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0062344849999999997, 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012030493999999999, 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84822 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888097+0000", 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888097+0000", 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888098+0000", 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6699999999999999e-07 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888099+0000", 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.928e-06 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888115+0000", 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5834000000000001e-05 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888121+0000", 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.451e-06 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.453 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84854 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888824+0000", 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0055072940000000003, 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011355262, 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84854 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888824+0000", 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888824+0000", 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888825+0000", 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.39e-07 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888827+0000", 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2539999999999999e-06 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888834+0000", 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4989999999999999e-06 2024-09-12T07:20:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888838+0000", 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.887e-06 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84581 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863417+0000", 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030914194999999998, 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036813178000000002, 2024-09-12T07:20:09.457 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84581 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863417+0000", 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863417+0000", 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863418+0000", 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.16e-07 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863419+0000", 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.221e-06 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863424+0000", 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863426+0000", 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.543e-06 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84613 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869568+0000", 2024-09-12T07:20:09.460 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024764077999999998, 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030710808999999999, 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84613 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869568+0000", 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869568+0000", 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869568+0000", 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0500000000000003e-07 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869570+0000", 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869576+0000", 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7509999999999997e-06 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869582+0000", 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4970000000000001e-06 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84645 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875426+0000", 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018905911000000001, 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024877836, 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84645 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875426+0000", 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875426+0000", 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875426+0000", 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.36e-07 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875427+0000", 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2529999999999999e-06 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875430+0000", 2024-09-12T07:20:09.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.055e-06 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875433+0000", 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5610000000000001e-06 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84791 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881535+0000", 2024-09-12T07:20:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012796531, 2024-09-12T07:20:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018794103, 2024-09-12T07:20:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84791 2024-09-12T07:20:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881535+0000", 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881535+0000", 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881535+0000", 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.41e-07 2024-09-12T07:20:09.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881537+0000", 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881543+0000", 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9399999999999999e-06 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881549+0000", 2024-09-12T07:20:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6550000000000001e-06 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84823 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888128+0000", 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0062031539999999998, 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012224024999999999, 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.471 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84823 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888128+0000", 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888128+0000", 2024-09-12T07:20:09.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888129+0000", 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.84e-07 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888131+0000", 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3489999999999999e-06 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888137+0000", 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8950000000000003e-06 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888142+0000", 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7010000000000004e-06 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.474 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84855 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888844+0000", 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0054872230000000003, 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011534078999999999, 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84855 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888844+0000", 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888844+0000", 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888845+0000", 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1200000000000003e-07 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888851+0000", 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3980000000000004e-06 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888858+0000", 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8680000000000001e-06 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888864+0000", 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.127e-06 2024-09-12T07:20:09.477 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84582 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863429+0000", 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030902559, 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036974269999999997, 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84582 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863429+0000", 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863429+0000", 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863429+0000", 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1500000000000001e-07 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863430+0000", 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.11e-06 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863434+0000", 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8550000000000004e-06 2024-09-12T07:20:09.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863438+0000", 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0029999999999996e-06 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84614 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869587+0000", 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024744644, 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030840287000000001, 2024-09-12T07:20:09.481 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84614 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869587+0000", 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869587+0000", 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869587+0000", 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.96e-07 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869590+0000", 2024-09-12T07:20:09.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0999999999999998e-06 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869596+0000", 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2040000000000002e-06 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869600+0000", 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4360000000000004e-06 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.485 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84646 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.485 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875436+0000", 2024-09-12T07:20:09.485 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018895945000000001, 2024-09-12T07:20:09.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025017251000000001, 2024-09-12T07:20:09.485 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.485 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.485 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.485 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.485 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.485 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84646 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875436+0000", 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875436+0000", 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875436+0000", 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1400000000000001e-07 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875440+0000", 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4299999999999999e-06 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875444+0000", 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4869999999999998e-06 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875447+0000", 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6840000000000001e-06 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84792 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881569+0000", 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012762506999999999, 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018920519, 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84792 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881569+0000", 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881569+0000", 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881570+0000", 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1699999999999998e-07 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881572+0000", 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1280000000000002e-06 2024-09-12T07:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881578+0000", 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5390000000000004e-06 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881584+0000", 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8699999999999997e-06 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84824 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888148+0000", 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.006183549, 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012395061000000001, 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84824 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888148+0000", 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888148+0000", 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888149+0000", 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3099999999999998e-07 2024-09-12T07:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888151+0000", 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888157+0000", 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0009999999999998e-06 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888162+0000", 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84856 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888871+0000", 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0054602009999999996, 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01171968, 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84856 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888871+0000", 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888871+0000", 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888872+0000", 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3500000000000002e-07 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888874+0000", 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.182e-06 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888880+0000", 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1199999999999999e-06 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888885+0000", 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8670000000000002e-06 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84583 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863444+0000", 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030887392999999999, 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037200916000000001, 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84583 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863444+0000", 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863444+0000", 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863445+0000", 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9400000000000001e-07 2024-09-12T07:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863447+0000", 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0590000000000001e-06 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863466+0000", 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9395e-05 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863471+0000", 2024-09-12T07:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8740000000000003e-06 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84615 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869606+0000", 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024725627999999999, 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031084694999999999, 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84615 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869606+0000", 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869606+0000", 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869606+0000", 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869608+0000", 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0140000000000001e-06 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869614+0000", 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.755e-06 2024-09-12T07:20:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869619+0000", 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6659999999999999e-06 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84647 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875450+0000", 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018881834, 2024-09-12T07:20:09.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025269208000000001, 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84647 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875450+0000", 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875450+0000", 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875450+0000", 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.23e-07 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875451+0000", 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2669999999999999e-06 2024-09-12T07:20:09.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875454+0000", 2024-09-12T07:20:09.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9699999999999999e-06 2024-09-12T07:20:09.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875457+0000", 2024-09-12T07:20:09.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.531e-06 2024-09-12T07:20:09.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.508 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84793 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881590+0000", 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012741194000000001, 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019152315999999999, 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84793 2024-09-12T07:20:09.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881590+0000", 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881590+0000", 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881591+0000", 2024-09-12T07:20:09.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2899999999999999e-07 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881593+0000", 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.818e-06 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881599+0000", 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9120000000000003e-06 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881603+0000", 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6299999999999997e-06 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84825 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888169+0000", 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0061625009999999999, 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012599308, 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84825 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888169+0000", 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888169+0000", 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888170+0000", 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9700000000000002e-07 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888172+0000", 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.289e-06 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888185+0000", 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3576e-05 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888192+0000", 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0929999999999999e-06 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84857 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.515 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888891+0000", 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0054403170000000001, 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011900665, 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84857 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888891+0000", 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888891+0000", 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888892+0000", 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888894+0000", 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.165e-06 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888901+0000", 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6440000000000003e-06 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888905+0000", 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8609999999999997e-06 2024-09-12T07:20:09.518 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84584 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863484+0000", 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030847864999999999, 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037334857999999999, 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.519 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84584 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863484+0000", 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863484+0000", 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863484+0000", 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863486+0000", 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9709999999999998e-06 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863494+0000", 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3580000000000003e-06 2024-09-12T07:20:09.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863498+0000", 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0309999999999999e-06 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84616 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869624+0000", 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024707468999999999, 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031219845999999999, 2024-09-12T07:20:09.522 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84616 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869624+0000", 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869624+0000", 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869625+0000", 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9800000000000004e-07 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869627+0000", 2024-09-12T07:20:09.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.052e-06 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869632+0000", 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.733e-06 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869639+0000", 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2820000000000002e-06 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84648 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875460+0000", 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018871744999999999, 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025409386999999999, 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84648 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.526 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875460+0000", 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875460+0000", 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875460+0000", 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1e-07 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875461+0000", 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1340000000000001e-06 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875464+0000", 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9849999999999998e-06 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875467+0000", 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5629999999999999e-06 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84794 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881610+0000", 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012721886, 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019297004, 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84794 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881610+0000", 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881610+0000", 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881610+0000", 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4499999999999998e-07 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881612+0000", 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.838e-06 2024-09-12T07:20:09.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881622+0000", 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6919999999999997e-06 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881627+0000", 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7150000000000003e-06 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84826 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888199+0000", 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0061325700000000004, 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012763485, 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84826 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.533 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888199+0000", 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888199+0000", 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888200+0000", 2024-09-12T07:20:09.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.63e-07 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888202+0000", 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3599999999999999e-06 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888210+0000", 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1810000000000001e-06 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888216+0000", 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5570000000000003e-06 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84858 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888912+0000", 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0054198129999999999, 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012100410000000001, 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84858 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888912+0000", 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888912+0000", 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888912+0000", 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888914+0000", 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.886e-06 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888920+0000", 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9429999999999997e-06 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888925+0000", 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9479999999999999e-06 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84585 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.539 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863502+0000", 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030830106999999999, 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037561211999999997, 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84585 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863502+0000", 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863502+0000", 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863502+0000", 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3799999999999999e-07 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863503+0000", 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3230000000000001e-06 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863506+0000", 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2049999999999998e-06 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863509+0000", 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6620000000000001e-06 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.542 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84617 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869644+0000", 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024687170000000001, 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031470106999999997, 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.543 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84617 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869644+0000", 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869644+0000", 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869645+0000", 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4299999999999999e-07 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869647+0000", 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7570000000000001e-06 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869653+0000", 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9320000000000001e-06 2024-09-12T07:20:09.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869663+0000", 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0669999999999999e-05 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84649 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875470+0000", 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018862001, 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025675209000000001, 2024-09-12T07:20:09.546 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84649 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875470+0000", 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875470+0000", 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875470+0000", 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.04e-07 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875471+0000", 2024-09-12T07:20:09.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0640000000000001e-06 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875474+0000", 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0120000000000001e-06 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875477+0000", 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6539999999999999e-06 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84795 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881634+0000", 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012697330999999999, 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019537354999999999, 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84795 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.550 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881634+0000", 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881634+0000", 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881635+0000", 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2500000000000001e-07 2024-09-12T07:20:09.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881637+0000", 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881644+0000", 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.725e-06 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881649+0000", 2024-09-12T07:20:09.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0760000000000002e-06 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84827 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888226+0000", 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0061058270000000003, 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012969045, 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.553 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84827 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888226+0000", 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888226+0000", 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888226+0000", 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1099999999999996e-07 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888229+0000", 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.17e-06 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888235+0000", 2024-09-12T07:20:09.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2480000000000001e-06 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888240+0000", 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0730000000000004e-06 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84859 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888930+0000", 2024-09-12T07:20:09.556 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0054012019999999999, 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012290809, 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84859 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888930+0000", 2024-09-12T07:20:09.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888930+0000", 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888931+0000", 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1600000000000001e-07 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888933+0000", 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8819999999999999e-06 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888939+0000", 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7470000000000002e-06 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888948+0000", 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7200000000000001e-06 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.559 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84586 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.863512+0000", 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.030819709000000001, 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037735106999999997, 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84586 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863512+0000", 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863512+0000", 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863512+0000", 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8500000000000002e-07 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863516+0000", 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1690000000000002e-06 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863520+0000", 2024-09-12T07:20:09.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.208e-06 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.863522+0000", 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5809999999999999e-06 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84618 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.869670+0000", 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.024661764999999999, 2024-09-12T07:20:09.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031602858999999997, 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84618 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869670+0000", 2024-09-12T07:20:09.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.869670+0000", 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.874971+0000", 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053006420000000004 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.874976+0000", 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4260000000000002e-06 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.874985+0000", 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2769999999999994e-06 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.874992+0000", 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0299999999999996e-06 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84650 3.17 3.18346f77 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.875479+0000", 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.018852183000000002, 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025816776, 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84650 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875479+0000", 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875479+0000", 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875480+0000", 2024-09-12T07:20:09.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0599999999999999e-07 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875481+0000", 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0440000000000001e-06 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875484+0000", 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0290000000000001e-06 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.875487+0000", 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4800000000000001e-06 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84796 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.881672+0000", 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.012659884, 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019656197, 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84796 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881672+0000", 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881672+0000", 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881672+0000", 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7899999999999998e-07 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881675+0000", 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5919999999999999e-06 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881683+0000", 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3599999999999996e-06 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.881689+0000", 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.573 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84828 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.574 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888248+0000", 2024-09-12T07:20:09.574 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0060838389999999997, 2024-09-12T07:20:09.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013127715, 2024-09-12T07:20:09.574 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84828 2024-09-12T07:20:09.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.574 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888248+0000", 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888248+0000", 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888248+0000", 2024-09-12T07:20:09.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.51e-07 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888251+0000", 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2579999999999998e-06 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888257+0000", 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3910000000000003e-06 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888262+0000", 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9289999999999997e-06 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:84860 3.17 3.18346f77 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.888955+0000", 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.005376736, 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012472119, 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:09.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 84860 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888955+0000", 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:09.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888955+0000", 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888955+0000", 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.01e-07 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888958+0000", 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2089999999999999e-06 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888964+0000", 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2009999999999996e-06 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.888969+0000", 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8010000000000003e-06 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:09.580 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 195 2024-09-12T07:20:09.581 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:09.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:20:09.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:09.588 INFO:teuthology.orchestra.run.smithi046.stdout:1291 trunc from 0x2ca8f31 to 0xc909623 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:1292 write 0x9cab54b thru 0x9cabf81 (0xa37 bytes) 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:1294 write 0x5f1bba6 thru 0x5f21ec1 (0x631c bytes) 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:1296 read 0xb34f994 thru 0xb35b74d (0xbdba bytes) 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:1298 write 0xc78de0a thru 0xc78ebe0 (0xdd7 bytes) 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:1299 write 0xb298bb4 thru 0xb2a3bb2 (0xafff bytes) 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:1301 read 0x189d1e2 thru 0x18a929f (0xc0be bytes) 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:1302 read 0xc865a6 thru 0xc8ee68 (0x88c3 bytes) 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:1303 read 0x19c01b3 thru 0x19c4574 (0x43c2 bytes) 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:1306 read 0x1efafb5 thru 0x1f09125 (0xe171 bytes) 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:1307 read 0x103a365 thru 0x103b546 (0x11e2 bytes) 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:1308 punch from 0x7c14947 to 0x7c18553, (0x3c0c bytes) 2024-09-12T07:20:09.589 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:1310 trunc from 0xc909623 to 0x7de791d 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:1311 write 0x7199390 thru 0x719eb90 (0x5801 bytes) 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:1312 writesame 0xa51c21d thru 0xa5219be data_size 0x3(0x57a2 bytes) 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:1314 read 0x30be1c3 thru 0x30bfd9e (0x1bdc bytes) 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:1316 read 0x34b754e thru 0x34c2a32 (0xb4e5 bytes) 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:1317 punch from 0x342c84b to 0x3438b01, (0xc2b6 bytes) 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:1318 write 0x595e876 thru 0x596857f (0x9d0a bytes) 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:1321 writesame 0x63a2704 thru 0x63aadc0 data_size 0x1(0x86bd bytes) 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:1322 trunc from 0xa5219bf to 0xa35feb6 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:1323 read 0x97af78d thru 0x97b1279 (0x1aed bytes) 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:1324 punch from 0xd74120 to 0xd7dc9e, (0x9b7e bytes) 2024-09-12T07:20:09.590 INFO:teuthology.orchestra.run.smithi046.stdout:1325 read 0x27a23e7 thru 0x27a7a42 (0x565c bytes) 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:1329 read 0x8bdb6ff thru 0x8bdd74a (0x204c bytes) 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:1330 writesame 0x41184e6 thru 0x4122e8d data_size 0x8(0xa9a8 bytes) 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:1331 read 0x7f6642e thru 0x7f6f95a (0x952d bytes) 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:1332 read 0x470f8cd thru 0x4711362 (0x1a96 bytes) 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:1334 writesame 0xebda69e thru 0xebe0beb data_size 0x1(0x654e bytes) 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:1335 read 0x66795e9 thru 0x667eed6 (0x58ee bytes) 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:1337 trunc from 0xebe0bec to 0x289bf78 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:1338 read 0xba961c thru 0xbb8c0b (0xf5f0 bytes) 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:1340 punch from 0x9dc2f3 to 0x9e09c2, (0x46cf bytes) 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:1342 write 0x78d5f42 thru 0x78d720c (0x12cb bytes) 2024-09-12T07:20:09.591 INFO:teuthology.orchestra.run.smithi046.stdout:1343 read 0x6816627 thru 0x6825099 (0xea73 bytes) 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:1344 write 0xd28beb thru 0xd2d9f2 (0x4e08 bytes) 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:1345 punch from 0x47843c9 to 0x478f133, (0xad6a bytes) 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:1346 punch from 0x14e1d45 to 0x14ec794, (0xaa4f bytes) 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:1348 read 0x4fd952a thru 0x4fe8c1d (0xf6f4 bytes) 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:1349 punch from 0x25866d2 to 0x2588e34, (0x2762 bytes) 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:1350 punch from 0x4f29c16 to 0x4f327c4, (0x8bae bytes) 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:1353 read 0xd9fa5e thru 0xda6920 (0x6ec3 bytes) 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:1354 write 0xccbeb68 thru 0xccbfdff (0x1298 bytes) 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:1355 write 0x2e025c7 thru 0x2e06daf (0x47e9 bytes) 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:1357 read 0xb6890d6 thru 0xb695b1d (0xca48 bytes) 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:1358 write 0x1994884 thru 0x199ffd5 (0xb752 bytes) 2024-09-12T07:20:09.592 INFO:teuthology.orchestra.run.smithi046.stdout:1360 write 0xbc20216 thru 0xbc25e90 (0x5c7b bytes) 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1362 write 0x1608c15 thru 0x16107cc (0x7bb8 bytes) 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1363 punch from 0x144cb4f to 0x14596b8, (0xcb69 bytes) 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1365 read 0x3e8df8a thru 0x3e908f9 (0x2970 bytes) 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1366 punch from 0xa65e11 to 0xa753e9, (0xf5d8 bytes) 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1368 read 0x30d062f thru 0x30d286d (0x223f bytes) 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1369 trunc from 0xccbfe00 to 0xe0f0ad0 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1370 writesame 0x82532f3 thru 0x825d1c7 data_size 0x1(0x9ed5 bytes) 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1371 punch from 0x251bd13 to 0x252ad43, (0xf030 bytes) 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1375 trunc from 0xe0f0ad0 to 0x308df88 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1376 read 0x2f836c2 thru 0x2f83d57 (0x696 bytes) 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1377 writesame 0x3efaee thru 0x3ff71b data_size 0x1a(0xfc2e bytes) 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1378 writesame 0x12183e0 thru 0x1225805 data_size 0x5(0xd426 bytes) 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1380 read 0x1b0735c thru 0x1b0cce8 (0x598d bytes) 2024-09-12T07:20:09.593 INFO:teuthology.orchestra.run.smithi046.stdout:1381 writesame 0x328c9a5 thru 0x328f6cf data_size 0x1(0x2d2b bytes) 2024-09-12T07:20:09.594 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:09.594 INFO:teuthology.orchestra.run.smithi046.stdout:1384 write 0x71d0232 thru 0x71d5958 (0x5727 bytes) 2024-09-12T07:20:09.594 INFO:teuthology.orchestra.run.smithi046.stdout:1386 write 0x712d115 thru 0x7136b63 (0x9a4f bytes) 2024-09-12T07:20:09.594 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:09.583+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:09.683 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:09.685+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:09.739 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:09.740 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:09.740 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:09.740 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:09.740 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:09.748 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:09.755 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:09.773 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:09.784 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:09.786+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:09.790 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:09.807 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:09.824 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:09.825 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:09.841 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:09.858 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:09.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:09.876 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:09.885 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:09.887+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:09.986 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:09.988+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:09.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:10.087 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:10.088+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:10.099 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:10.100 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:10.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:10.192 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:10.194+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:10.194 INFO:teuthology.orchestra.run.smithi046.stdout:1388 punch from 0x7051606 to 0x7057f53, (0x694d bytes) 2024-09-12T07:20:10.195 INFO:teuthology.orchestra.run.smithi046.stdout:1389 punch from 0x40aad2a to 0x40b040a, (0x56e0 bytes) 2024-09-12T07:20:10.195 INFO:teuthology.orchestra.run.smithi046.stdout:1390 write 0x5e73de9 thru 0x5e7927f (0x5497 bytes) 2024-09-12T07:20:10.200 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:20:10.208 INFO:teuthology.orchestra.run.smithi046.stdout:1391 trunc from 0x71d5959 to 0xbc74970 2024-09-12T07:20:10.216 INFO:teuthology.orchestra.run.smithi046.stdout:1392 writesame 0xa599891 thru 0xa5a4e0a data_size 0x3a(0xb57a bytes) 2024-09-12T07:20:10.225 INFO:teuthology.orchestra.run.smithi046.stdout:1394 write 0xa5228b4 thru 0xa52baf3 (0x9240 bytes) 2024-09-12T07:20:10.228 INFO:teuthology.orchestra.run.smithi046.stdout:1395 write 0x279886f thru 0x279be2c (0x35be bytes) 2024-09-12T07:20:10.234 INFO:teuthology.orchestra.run.smithi046.stdout:1396 read 0x2c5533f thru 0x2c58fa3 (0x3c65 bytes) 2024-09-12T07:20:10.235 INFO:teuthology.orchestra.run.smithi046.stdout:1397 writesame 0xb1a6609 thru 0xb1b4d68 data_size 0x30(0xe760 bytes) 2024-09-12T07:20:10.244 INFO:teuthology.orchestra.run.smithi046.stdout:1398 read 0xb225579 thru 0xb228dfa (0x3882 bytes) 2024-09-12T07:20:10.252 INFO:teuthology.orchestra.run.smithi046.stdout:1399 write 0xd1385d7 thru 0xd144642 (0xc06c bytes) 2024-09-12T07:20:10.264 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:10.264 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:10.264 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:10.264 INFO:teuthology.orchestra.run.smithi046.stdout:1403 write 0x20147b4 thru 0x2021a86 (0xd2d3 bytes) 2024-09-12T07:20:10.272 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:10.273 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:10.279 INFO:teuthology.orchestra.run.smithi046.stdout:1404 trunc from 0xd144643 to 0x732440e 2024-09-12T07:20:10.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:10.288 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:10.290+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:10.292 INFO:teuthology.orchestra.run.smithi046.stdout:1405 write 0x6950d01 thru 0x695b433 (0xa733 bytes) 2024-09-12T07:20:10.302 INFO:teuthology.orchestra.run.smithi046.stdout:1407 read 0x122fd02 thru 0x1234a78 (0x4d77 bytes) 2024-09-12T07:20:10.303 INFO:teuthology.orchestra.run.smithi046.stdout:1408 read 0x32c3b18 thru 0x32c5821 (0x1d0a bytes) 2024-09-12T07:20:10.303 INFO:teuthology.orchestra.run.smithi046.stdout:1409 read 0x9cb5d thru 0x9fa66 (0x2f0a bytes) 2024-09-12T07:20:10.312 INFO:teuthology.orchestra.run.smithi046.stdout:1410 write 0xa055418 thru 0xa06129b (0xbe84 bytes) 2024-09-12T07:20:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:1411 write 0xc7aea7c thru 0xc7ba633 (0xbbb8 bytes) 2024-09-12T07:20:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:1413 punch from 0xa426b62 to 0xa42c830, (0x5cce bytes) 2024-09-12T07:20:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:1414 writesame 0x4b4685e thru 0x4b47226 data_size 0x5(0x9c9 bytes) 2024-09-12T07:20:10.347 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:10.347 INFO:teuthology.orchestra.run.smithi046.stdout:1419 punch from 0x645657a to 0x645b86d, (0x52f3 bytes) 2024-09-12T07:20:10.360 INFO:teuthology.orchestra.run.smithi046.stdout:1420 trunc from 0xc7ba634 to 0x101f7e9 2024-09-12T07:20:10.379 INFO:teuthology.orchestra.run.smithi046.stdout:1421 punch from 0x5c4bbb to 0x5cc7d9, (0x7c1e bytes) 2024-09-12T07:20:10.385 INFO:teuthology.orchestra.run.smithi046.stdout:1422 write 0x21ed117 thru 0x21f83ab (0xb295 bytes) 2024-09-12T07:20:10.390 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:10.391+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:10.409 INFO:teuthology.orchestra.run.smithi046.stdout:1423 write 0xdf741a0 thru 0xdf79a04 (0x5865 bytes) 2024-09-12T07:20:10.414 INFO:teuthology.orchestra.run.smithi046.stdout:1424 write 0x195380d thru 0x195b417 (0x7c0b bytes) 2024-09-12T07:20:10.419 INFO:teuthology.orchestra.run.smithi046.stdout:1425 read 0xc53518a thru 0xc53f3ba (0xa231 bytes) 2024-09-12T07:20:10.428 INFO:teuthology.orchestra.run.smithi046.stdout:1427 trunc from 0xdf79a05 to 0x639e46a 2024-09-12T07:20:10.438 INFO:teuthology.orchestra.run.smithi046.stdout:1428 write 0x42f269e thru 0x42f663b (0x3f9e bytes) 2024-09-12T07:20:10.442 INFO:teuthology.orchestra.run.smithi046.stdout:1429 punch from 0x2cdd5b6 to 0x2ce3182, (0x5bcc bytes) 2024-09-12T07:20:10.450 INFO:teuthology.orchestra.run.smithi046.stdout:1431 write 0x85fd5ab thru 0x8606f1f (0x9975 bytes) 2024-09-12T07:20:10.455 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:10.455 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:10.458 INFO:teuthology.orchestra.run.smithi046.stdout:1432 trunc from 0x8606f20 to 0x54be6d3 2024-09-12T07:20:10.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:10.477 INFO:teuthology.orchestra.run.smithi046.stdout:1433 trunc from 0x54be6d3 to 0xc37fdbd 2024-09-12T07:20:10.483 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:10.483 INFO:teuthology.orchestra.run.smithi046.stdout:1437 read 0xab95ea thru 0xac86ee (0xf105 bytes) 2024-09-12T07:20:10.484 INFO:teuthology.orchestra.run.smithi046.stdout:1439 read 0xab126ac thru 0xab17a80 (0x53d5 bytes) 2024-09-12T07:20:10.491 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:10.492+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:10.491 INFO:teuthology.orchestra.run.smithi046.stdout:1440 trunc from 0xc37fdbd to 0x5c099ef 2024-09-12T07:20:10.497 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:10.497 INFO:teuthology.orchestra.run.smithi046.stdout:1443 punch from 0xd0256c to 0xd0a771, (0x8205 bytes) 2024-09-12T07:20:10.497 INFO:teuthology.orchestra.run.smithi046.stdout:1444 read 0x50d17d thru 0x510610 (0x3494 bytes) 2024-09-12T07:20:10.506 INFO:teuthology.orchestra.run.smithi046.stdout:1446 trunc from 0x5c099ef to 0xd8db395 2024-09-12T07:20:10.512 INFO:teuthology.orchestra.run.smithi046.stdout:1448 writesame 0x56f85bd thru 0x56fc473 data_size 0x41(0x3eb7 bytes) 2024-09-12T07:20:10.517 INFO:teuthology.orchestra.run.smithi046.stdout:1449 read 0x894b23c thru 0x895a186 (0xef4b bytes) 2024-09-12T07:20:10.523 INFO:teuthology.orchestra.run.smithi046.stdout:1450 writesame 0xd8f0d93 thru 0xd8f3ec5 data_size 0xb(0x3133 bytes) 2024-09-12T07:20:10.523 INFO:teuthology.orchestra.run.smithi046.stdout:1451 writesame 0x58bb585 thru 0x58c4369 data_size 0x5(0x8de5 bytes) 2024-09-12T07:20:10.544 INFO:teuthology.orchestra.run.smithi046.stdout:1452 trunc from 0xd8f3ec6 to 0x10888ce 2024-09-12T07:20:10.563 INFO:teuthology.orchestra.run.smithi046.stdout:1453 write 0x34e249c thru 0x34e7680 (0x51e5 bytes) 2024-09-12T07:20:10.569 INFO:teuthology.orchestra.run.smithi046.stdout:1454 punch from 0x298dde8 to 0x2998051, (0xa269 bytes) 2024-09-12T07:20:10.569 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:10.580 INFO:teuthology.orchestra.run.smithi046.stdout:1456 trunc from 0x34e7681 to 0xd3bbf90 2024-09-12T07:20:10.586 INFO:teuthology.orchestra.run.smithi046.stdout:1457 writesame 0xbfe0904 thru 0xbfe4bc9 data_size 0x2a(0x42c6 bytes) 2024-09-12T07:20:10.591 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:10.593+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:10.593 INFO:teuthology.orchestra.run.smithi046.stdout:1458 write 0xac2ddd5 thru 0xac2f975 (0x1ba1 bytes) 2024-09-12T07:20:10.600 INFO:teuthology.orchestra.run.smithi046.stdout:1460 clone 12 order 20 su 65536 sc 9 2024-09-12T07:20:10.605 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:10.605 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:10.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:10.692 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:10.695+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:10.760 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:10.761 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:10.769 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:10.812 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:10.814+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:10.876 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:10.876 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:10.885 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:10.894 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:10.896+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:10.995 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:10.996+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:11.031 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:11.032 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:11.040 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:11.095 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:11.097+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:11.187 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:11.187 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:11.189 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:20:11.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-12T07:20:11.190 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:20:11.200 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:11.200 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:11.201+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:11.297 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:11.299+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:11.315 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:11.315 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:11.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:11.384 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-12T07:20:11.408 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:11.410+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:11.473 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:11.473 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:11.481 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:11.523 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:11.524+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:11.600 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:11.601+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:11.628 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:11.628 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:11.636 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:11.643 INFO:teuthology.orchestra.run.smithi046.stdout:1462 read 0x3a3e132 thru 0x3a47e40 (0x9d0f bytes) 2024-09-12T07:20:11.644 INFO:teuthology.orchestra.run.smithi046.stdout:1463 write 0x2fbd645 thru 0x2fcaa19 (0xd3d5 bytes) 2024-09-12T07:20:11.655 INFO:teuthology.orchestra.run.smithi046.stdout:1464 write 0xace6fbb thru 0xacf4c00 (0xdc46 bytes) 2024-09-12T07:20:11.660 INFO:teuthology.orchestra.run.smithi046.stdout:1465 write 0x2a4aa2c thru 0x2a55160 (0xa735 bytes) 2024-09-12T07:20:11.664 INFO:teuthology.orchestra.run.smithi046.stdout:1466 punch from 0xc4989f9 to 0xc49e596, (0x5b9d bytes) 2024-09-12T07:20:11.664 INFO:teuthology.orchestra.run.smithi046.stdout:1467 read 0xb7d7633 thru 0xb7d8601 (0xfcf bytes) 2024-09-12T07:20:11.664 INFO:teuthology.orchestra.run.smithi046.stdout:1468 punch from 0x9faee12 to 0x9fb0041, (0x122f bytes) 2024-09-12T07:20:11.664 INFO:teuthology.orchestra.run.smithi046.stdout:1470 read 0x3bc2029 thru 0x3bcbea4 (0x9e7c bytes) 2024-09-12T07:20:11.664 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:11.664 INFO:teuthology.orchestra.run.smithi046.stdout:1473 writesame 0x8f9f31d thru 0x8fae449 data_size 0x1(0xf12d bytes) 2024-09-12T07:20:11.698 INFO:teuthology.orchestra.run.smithi046.stdout:1476 read 0x4190650 thru 0x4192b52 (0x2503 bytes) 2024-09-12T07:20:11.698 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:11.698 INFO:teuthology.orchestra.run.smithi046.stdout:1479 read 0xc834a19 thru 0xc839d50 (0x5338 bytes) 2024-09-12T07:20:11.698 INFO:teuthology.orchestra.run.smithi046.stdout:1480 write 0xb8e4b23 thru 0xb8eb1f0 (0x66ce bytes) 2024-09-12T07:20:11.701 INFO:teuthology.orchestra.run.smithi046.stdout:1481 read 0x11765a6 thru 0x1183f45 (0xd9a0 bytes) 2024-09-12T07:20:11.701 INFO:teuthology.orchestra.run.smithi046.stdout:1482 read 0xa856f92 thru 0xa85c402 (0x5471 bytes) 2024-09-12T07:20:11.709 INFO:teuthology.orchestra.run.smithi046.stdout:1483 trunc from 0xd3bbf90 to 0x5f84154 2024-09-12T07:20:11.720 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:11.722+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:11.737 INFO:teuthology.orchestra.run.smithi046.stdout:1486 trunc from 0x5f84154 to 0x114a91c 2024-09-12T07:20:11.765 INFO:teuthology.orchestra.run.smithi046.stdout:1487 punch from 0x100896e to 0x101436c, (0xb9fe bytes) 2024-09-12T07:20:11.766 INFO:teuthology.orchestra.run.smithi046.stdout:1488 punch from 0x4852a to 0x49f05, (0x19db bytes) 2024-09-12T07:20:11.766 INFO:teuthology.orchestra.run.smithi046.stdout:1492 read 0x6b06d6 thru 0x6bcbbc (0xc4e7 bytes) 2024-09-12T07:20:11.781 INFO:teuthology.orchestra.run.smithi046.stdout:1494 writesame 0xa5b281b thru 0xa5b602f data_size 0x7(0x3815 bytes) 2024-09-12T07:20:11.788 INFO:teuthology.orchestra.run.smithi046.stdout:1495 write 0xbff0c43 thru 0xbff7e4d (0x720b bytes) 2024-09-12T07:20:11.791 INFO:teuthology.orchestra.run.smithi046.stdout:1496 read 0x57c5d1a thru 0x57d46d3 (0xe9ba bytes) 2024-09-12T07:20:11.791 INFO:teuthology.orchestra.run.smithi046.stdout:1497 punch from 0x90da5c6 to 0x90e4fd9, (0xaa13 bytes) 2024-09-12T07:20:11.791 INFO:teuthology.orchestra.run.smithi046.stdout:1498 write 0x307a8bb thru 0x308027c (0x59c2 bytes) 2024-09-12T07:20:11.795 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:11.795 INFO:teuthology.orchestra.run.smithi046.stdout:1500 punch from 0x460f79b to 0x46138a7, (0x410c bytes) 2024-09-12T07:20:11.795 INFO:teuthology.orchestra.run.smithi046.stdout:1501 write 0x5477037 thru 0x547c1aa (0x5174 bytes) 2024-09-12T07:20:11.799 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:11.799 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:11.799 INFO:teuthology.orchestra.run.smithi046.stdout:1502 read 0x5e6ef72 thru 0x5e7dc83 (0xed12 bytes) 2024-09-12T07:20:11.800 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:11.802+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:11.808 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:11.808 INFO:teuthology.orchestra.run.smithi046.stdout:1504 trunc from 0xbff7e4e to 0x4132b8c 2024-09-12T07:20:11.828 INFO:teuthology.orchestra.run.smithi046.stdout:1509 write 0x876c174 thru 0x8778398 (0xc225 bytes) 2024-09-12T07:20:11.844 INFO:teuthology.orchestra.run.smithi046.stdout:1511 write 0xd4bd563 thru 0xd4cd0b1 (0xfb4f bytes) 2024-09-12T07:20:11.848 INFO:teuthology.orchestra.run.smithi046.stdout:1512 write 0xaa2d83c thru 0xaa313a9 (0x3b6e bytes) 2024-09-12T07:20:11.852 INFO:teuthology.orchestra.run.smithi046.stdout:1513 punch from 0x5ca8356 to 0x5cb2f9d, (0xac47 bytes) 2024-09-12T07:20:11.852 INFO:teuthology.orchestra.run.smithi046.stdout:1516 write 0x606b748 thru 0x607786a (0xc123 bytes) 2024-09-12T07:20:11.857 INFO:teuthology.orchestra.run.smithi046.stdout:1517 read 0x9814be0 thru 0x982325a (0xe67b bytes) 2024-09-12T07:20:11.857 INFO:teuthology.orchestra.run.smithi046.stdout:1520 read 0xf54c34 thru 0xf5b244 (0x6611 bytes) 2024-09-12T07:20:11.858 INFO:teuthology.orchestra.run.smithi046.stdout:1521 punch from 0x72ec5bf to 0x72f58d8, (0x9319 bytes) 2024-09-12T07:20:11.868 INFO:teuthology.orchestra.run.smithi046.stdout:1523 trunc from 0xd4cd0b2 to 0x413f3bb 2024-09-12T07:20:11.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:11.895 INFO:teuthology.orchestra.run.smithi046.stdout:1524 write 0x972a364 thru 0x9732e5c (0x8af9 bytes) 2024-09-12T07:20:11.900 INFO:teuthology.orchestra.run.smithi046.stdout:1525 read 0x2cf1323 thru 0x2cf8732 (0x7410 bytes) 2024-09-12T07:20:11.900 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:11.900 INFO:teuthology.orchestra.run.smithi046.stdout:1529 punch from 0x1755e69 to 0x1759b34, (0x3ccb bytes) 2024-09-12T07:20:11.900 INFO:teuthology.orchestra.run.smithi046.stdout:1530 write 0x740013b thru 0x7405a9e (0x5964 bytes) 2024-09-12T07:20:11.901 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:11.903+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:11.913 INFO:teuthology.orchestra.run.smithi046.stdout:1531 write 0xc9b58e4 thru 0xc9b7643 (0x1d60 bytes) 2024-09-12T07:20:11.915 INFO:teuthology.orchestra.run.smithi046.stdout:1532 writesame 0x27e0bd7 thru 0x27e76b2 data_size 0xe(0x6adc bytes) 2024-09-12T07:20:11.919 INFO:teuthology.orchestra.run.smithi046.stdout:1534 write 0xa46a9ed thru 0xa479af5 (0xf109 bytes) 2024-09-12T07:20:11.933 INFO:teuthology.orchestra.run.smithi046.stdout:1535 write 0xdb7aade thru 0xdb8612b (0xb64e bytes) 2024-09-12T07:20:11.941 INFO:teuthology.orchestra.run.smithi046.stdout:1536 writesame 0x29f9b11 thru 0x2a04e1e data_size 0x52(0xb30e bytes) 2024-09-12T07:20:11.959 INFO:teuthology.orchestra.run.smithi046.stdout:1538 trunc from 0xdb8612c to 0x540d079 2024-09-12T07:20:11.965 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:11.965 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:11.971 INFO:teuthology.orchestra.run.smithi046.stdout:1539 trunc from 0x540d079 to 0x6f2f874 2024-09-12T07:20:11.974 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:11.985 INFO:teuthology.orchestra.run.smithi046.stdout:1542 write 0xac10cb8 thru 0xac14a76 (0x3dbf bytes) 2024-09-12T07:20:11.988 INFO:teuthology.orchestra.run.smithi046.stdout:1544 writesame 0x1a6607f thru 0x1a66375 data_size 0xb(0x2f7 bytes) 2024-09-12T07:20:11.994 INFO:teuthology.orchestra.run.smithi046.stdout:1546 trunc from 0xac14a77 to 0x7a2c64b 2024-09-12T07:20:12.003 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:12.003 INFO:teuthology.orchestra.run.smithi046.stdout:1549 read 0x39b5331 thru 0x39b8280 (0x2f50 bytes) 2024-09-12T07:20:12.003 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:12.014 INFO:teuthology.orchestra.run.smithi046.stdout:1551 write 0xd4ff13f thru 0xd4ffce2 (0xba4 bytes) 2024-09-12T07:20:12.015 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:12.017+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:12.017 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:12.017 INFO:teuthology.orchestra.run.smithi046.stdout:1553 read 0xad9cedd thru 0xadaba66 (0xeb8a bytes) 2024-09-12T07:20:12.017 INFO:teuthology.orchestra.run.smithi046.stdout:1554 write 0x3a2fa52 thru 0x3a37c7e (0x822d bytes) 2024-09-12T07:20:12.025 INFO:teuthology.orchestra.run.smithi046.stdout:1555 writesame 0x2694192 thru 0x26a3e77 data_size 0x1(0xfce6 bytes) 2024-09-12T07:20:12.029 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:12.029 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:12.048 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:12.048 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:12.049 INFO:teuthology.orchestra.run.smithi046.stdout:1556 clone 13 order 19 su 262144 sc 11 2024-09-12T07:20:12.064 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:12.081 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:12.098 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:12.103 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:12.105+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:12.115 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:12.125 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:12.125 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:12.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:12.125 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:12.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.125 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.125 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:12.125 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.644053048, 2024-09-12T07:20:12.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:20:12.125 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:20:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:20:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.644026168, 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:20:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:20:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:20:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:20:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.644002148, 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:20:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.643974502000001, 2024-09-12T07:20:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:20:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:20:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:20:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:20:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.643951039999999, 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.643928513000001, 2024-09-12T07:20:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:20:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:20:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:20:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.64390671, 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.63727293, 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:20:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:20:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:20:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:20:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.637252922, 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:20:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:20:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:20:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:20:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:20:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.63076989, 2024-09-12T07:20:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:20:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.168 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:20:12.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:20:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:20:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.630734603000001, 2024-09-12T07:20:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:20:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:20:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.63071197, 2024-09-12T07:20:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:20:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:20:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.630657915, 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:20:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.630638974, 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.630627037, 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.630615422, 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:20:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.623602814, 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:20:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.623579099000001, 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:20:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:20:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:20:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:20:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 12.623448242, 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:20:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 8.6597774520000002, 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:20:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:20:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.218 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:12.219 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:12.220 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:12.221 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:12.221 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:12.222 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:12.221+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:12.222 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:12.223 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.0 to 1 (10000) 2024-09-12T07:20:12.223 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-12T07:20:12.261 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone12 from 0xd4ffce3 (overlap 0x114a91c) to 0x375505 2024-09-12T07:20:12.321 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:12.322+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:12.342 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-12T07:20:12.376 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:12.377 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:12.377 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:12.377 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:12.385 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:12.421 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:12.423+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:12.522 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:12.524+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:12.533 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:12.534 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:12.541 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:12.623 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:12.625+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:12.630 INFO:teuthology.orchestra.run.smithi046.stdout:1558 punch from 0x1fa84ae to 0x1fab286, (0x2dd8 bytes) 2024-09-12T07:20:12.636 INFO:teuthology.orchestra.run.smithi046.stdout:1559 read 0x414c290 thru 0x4151bd5 (0x5946 bytes) 2024-09-12T07:20:12.636 INFO:teuthology.orchestra.run.smithi046.stdout:1560 read 0x3589136 thru 0x3590f2b (0x7df6 bytes) 2024-09-12T07:20:12.643 INFO:teuthology.orchestra.run.smithi046.stdout:1561 write 0xe87f9f6 thru 0xe882870 (0x2e7b bytes) 2024-09-12T07:20:12.646 INFO:teuthology.orchestra.run.smithi046.stdout:1563 read 0x1996380 thru 0x1998ab0 (0x2731 bytes) 2024-09-12T07:20:12.646 INFO:teuthology.orchestra.run.smithi046.stdout:1564 punch from 0x42160c2 to 0x421a2e6, (0x4224 bytes) 2024-09-12T07:20:12.646 INFO:teuthology.orchestra.run.smithi046.stdout:1565 write 0x3d75c7d thru 0x3d790bf (0x3443 bytes) 2024-09-12T07:20:12.649 INFO:teuthology.orchestra.run.smithi046.stdout:1566 punch from 0x7a6b87d to 0x7a7518f, (0x9912 bytes) 2024-09-12T07:20:12.649 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:12.649 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:12.649 INFO:teuthology.orchestra.run.smithi046.stdout:1571 write 0x40ce6fb thru 0x40d64c3 (0x7dc9 bytes) 2024-09-12T07:20:12.651 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:20:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 20.122306173999998, 2024-09-12T07:20:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:20:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:20:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:20:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.270508508000001, 2024-09-12T07:20:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:20:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.270385231000001, 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:20:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.662 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.662 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.662 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:12.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:20:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:20:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.270181839999999, 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:20:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.270046013, 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.269719357, 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.269680469000001, 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:20:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.269313595, 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.269178739000001, 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:20:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:20:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:20:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:20:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:20:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:20:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.268808848000001, 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:20:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:20:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.268618650000001, 2024-09-12T07:20:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:20:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:20:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.268398571000001, 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:20:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.268254905999999, 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:20:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.268028734, 2024-09-12T07:20:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.267828376000001, 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.267420492999999, 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.266954610999999, 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.265352400999999, 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:20:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.264818366, 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.008620296, 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:20:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.741 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.741 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.741 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:12.741 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:12.741 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:12.743 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:12.744 INFO:teuthology.orchestra.run.smithi046.stdout:1572 writesame 0x92153e4 thru 0x921e1d0 data_size 0x21(0x8ded bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1573 punch from 0xb25fced to 0xb260f1b, (0x122e bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1574 write 0x9b389c6 thru 0x9b3e61a (0x5c55 bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1575 writesame 0xe4b52f2 thru 0xe4bfba4 data_size 0x1(0xa8b3 bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1576 read 0x4342063 thru 0x434d48d (0xb42b bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1577 read 0xdbec044 thru 0xdbed144 (0x1101 bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1578 write 0xa14c7c9 thru 0xa1591b2 (0xc9ea bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1579 read 0x125647 thru 0x12e150 (0x8b0a bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1580 punch from 0xa0c07f to 0xa17639, (0xb5ba bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1581 read 0xa92320c thru 0xa930c6b (0xda60 bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1582 write 0x8ae56ba thru 0x8ae6417 (0xd5e bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1585 write 0x5ca87ed thru 0x5ca9bdc (0x13f0 bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1586 read 0x4b53b7a thru 0x4b55e34 (0x22bb bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1588 write 0x459c18a thru 0x45a0f80 (0x4df7 bytes) 2024-09-12T07:20:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:1589 read 0xd48740c thru 0xd496fbe (0xfbb3 bytes) 2024-09-12T07:20:12.746 INFO:teuthology.orchestra.run.smithi046.stdout:1590 write 0x89b302d thru 0x89be735 (0xb709 bytes) 2024-09-12T07:20:12.746 INFO:teuthology.orchestra.run.smithi046.stdout:1591 read 0xb08562a thru 0xb08fea8 (0xa87f bytes) 2024-09-12T07:20:12.746 INFO:teuthology.orchestra.run.smithi046.stdout:1592 read 0x319e4df thru 0x31ab152 (0xcc74 bytes) 2024-09-12T07:20:12.746 INFO:teuthology.orchestra.run.smithi046.stdout:1593 trunc from 0xe882871 to 0x6a0a1e0 2024-09-12T07:20:12.746 INFO:teuthology.orchestra.run.smithi046.stdout:1594 read 0x5ddc612 thru 0x5de8879 (0xc268 bytes) 2024-09-12T07:20:12.746 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:12.746 INFO:teuthology.orchestra.run.smithi046.stdout:1596 write 0x679a27c thru 0x67a6cb7 (0xca3c bytes) 2024-09-12T07:20:12.746 INFO:teuthology.orchestra.run.smithi046.stdout:1597 clone 14 order 22 su 32768 sc 12 2024-09-12T07:20:12.746 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:12.743+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:12.843 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:12.844+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:12.890 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:12.890 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:12.890 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:12.890 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:12.898 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:12.944 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:12.946+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:13.005 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:13.005 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:13.005 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:13.005 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:13.005 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:13.013 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:13.058 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:13.060+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:13.121 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5733625979999992, 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:20:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:20:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.573342061, 2024-09-12T07:20:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:20:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:20:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:20:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:20:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:20:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5733220580000005, 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5733021090000001, 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5732812410000001, 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5732595400000005, 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5732391000000003, 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.573211379, 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5731884439999995, 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5731681440000003, 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5731478679999995, 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:20:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5731277089999995, 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:20:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5729905110000004, 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:20:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5729678630000006, 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:20:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:20:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:20:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68088 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 393216~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5665251040000001, 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10346986399999999, 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68088 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555923+0000", 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.891e-06 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555929+0000", 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0549999999999996e-06 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555934+0000", 2024-09-12T07:20:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6419999999999998e-06 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659060+0000", 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103125862 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659139+0000", 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9450000000000007e-05 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659391+0000", 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025158400000000001 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5665058770000009, 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:20:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5662762400000005, 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5593525489999998, 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5589929659999999, 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5589129800000006, 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:20:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:20:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.210 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.210 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.210 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:13.210 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:13.210 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:13.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:20:13.211 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.1 to 1 (10000) 2024-09-12T07:20:13.212 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:13.212+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:13.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:13.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-12T07:20:13.285 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone13 from 0x6a0a1e0 (overlap 0x6a0a1e0) to 0x2a6a726 2024-09-12T07:20:13.312 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:13.313+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:13.371 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:13.371 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:13.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:88528 3.b e53/46 3:d7d0ac0d:::rbd_object_map.1160d8a91ade:head v 53'108, mlcod=53'105)", 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:13.373133+0000", 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.00045354299999999999, 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047603800000000003, 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.373133+0000", 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.373133+0000", 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.373134+0000", 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6299999999999993e-07 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.373142+0000", 2024-09-12T07:20:13.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4810000000000006e-06 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.373157+0000", 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.499e-05 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.373165+0000", 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9740000000000002e-06 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.373207+0000", 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1356000000000002e-05 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.373220+0000", 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3756e-05 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 1 2024-09-12T07:20:13.375 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:13.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:13.425 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:13.426+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:13.469 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-12T07:20:13.477 INFO:teuthology.orchestra.run.smithi046.stdout:1598 punch from 0x4777b74 to 0x477ff1d, (0x83a9 bytes) 2024-09-12T07:20:13.481 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:13.481 INFO:teuthology.orchestra.run.smithi046.stdout:1600 writesame 0x3a01066 thru 0x3a07576 data_size 0x1(0x6511 bytes) 2024-09-12T07:20:13.514 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:13.515+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:13.518 INFO:teuthology.orchestra.run.smithi046.stdout:1601 trunc from 0x6a0a1e0 to 0xb2b05d9 2024-09-12T07:20:13.524 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:13.524 INFO:teuthology.orchestra.run.smithi046.stdout:1603 punch from 0x77df7a9 to 0x77e568e, (0x5ee5 bytes) 2024-09-12T07:20:13.524 INFO:teuthology.orchestra.run.smithi046.stdout:1604 write 0x32040a1 thru 0x321000c (0xbf6c bytes) 2024-09-12T07:20:13.546 INFO:teuthology.orchestra.run.smithi046.stdout:1605 punch from 0xb1312f to 0xb16216, (0x30e7 bytes) 2024-09-12T07:20:13.546 INFO:teuthology.orchestra.run.smithi046.stdout:1607 read 0x697cb29 thru 0x697fa9e (0x2f76 bytes) 2024-09-12T07:20:13.546 INFO:teuthology.orchestra.run.smithi046.stdout:1608 write 0x61543ff thru 0x6155a16 (0x1618 bytes) 2024-09-12T07:20:13.546 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:13.546 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:13.550 INFO:teuthology.orchestra.run.smithi046.stdout:1609 write 0x3d1b0b1 thru 0x3d268ca (0xb81a bytes) 2024-09-12T07:20:13.559 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:13.579 INFO:teuthology.orchestra.run.smithi046.stdout:1610 read 0x3dcbefe thru 0x3dd0613 (0x4716 bytes) 2024-09-12T07:20:13.579 INFO:teuthology.orchestra.run.smithi046.stdout:1613 write 0x195624c thru 0x1957472 (0x1227 bytes) 2024-09-12T07:20:13.599 INFO:teuthology.orchestra.run.smithi046.stdout:1614 writesame 0x48a7888 thru 0x48b3aa4 data_size 0x7(0xc21d bytes) 2024-09-12T07:20:13.615 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:13.617+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:13.635 INFO:teuthology.orchestra.run.smithi046.stdout:1615 writesame 0xe7ee916 thru 0xe7f1235 data_size 0x20(0x2920 bytes) 2024-09-12T07:20:13.641 INFO:teuthology.orchestra.run.smithi046.stdout:1616 trunc from 0xe7f1236 to 0x95698f3 2024-09-12T07:20:13.647 INFO:teuthology.orchestra.run.smithi046.stdout:1617 read 0x818db22 thru 0x8194ba1 (0x7080 bytes) 2024-09-12T07:20:13.647 INFO:teuthology.orchestra.run.smithi046.stdout:1618 write 0x15d94c9 thru 0x15e55e0 (0xc118 bytes) 2024-09-12T07:20:13.667 INFO:teuthology.orchestra.run.smithi046.stdout:1619 write 0x7ccb3b6 thru 0x7cd4d42 (0x998d bytes) 2024-09-12T07:20:13.673 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:13.673 INFO:teuthology.orchestra.run.smithi046.stdout:1622 read 0x622fb78 thru 0x623a357 (0xa7e0 bytes) 2024-09-12T07:20:13.673 INFO:teuthology.orchestra.run.smithi046.stdout:1623 write 0x228e90b thru 0x229c2bb (0xd9b1 bytes) 2024-09-12T07:20:13.696 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82749 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 429056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0527452080000002, 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055052166, 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82749 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0800000000000001e-07 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644459+0000", 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1769999999999999e-06 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644462+0000", 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8779999999999998e-06 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644465+0000", 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6509999999999999e-06 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699279+0000", 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054813832999999999 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699359+0000", 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0150999999999999e-05 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699510+0000", 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015026800000000001 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82750 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 432128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0527323239999999, 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055234722, 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82750 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644471+0000", 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.17e-07 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644472+0000", 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.19e-06 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644482+0000", 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0332e-05 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644489+0000", 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7039999999999997e-06 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699549+0000", 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055060588000000001 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699592+0000", 2024-09-12T07:20:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2707000000000003e-05 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699705+0000", 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112984 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82751 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 436224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0527049699999997, 2024-09-12T07:20:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055415691000000003, 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82751 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644846+0000", 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034807600000000001 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644850+0000", 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1649999999999999e-06 2024-09-12T07:20:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644860+0000", 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6949999999999995e-06 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644866+0000", 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3640000000000004e-06 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699735+0000", 2024-09-12T07:20:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054868888999999997 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699776+0000", 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0722000000000001e-05 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699913+0000", 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013778 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82752 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 439296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0523272270000001, 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226917, 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82752 2024-09-12T07:20:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644878+0000", 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7850000000000001e-06 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644882+0000", 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9990000000000002e-06 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644885+0000", 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.965e-06 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699952+0000", 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067243000000002 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699989+0000", 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7450999999999998e-05 2024-09-12T07:20:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700102+0000", 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113124 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82753 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 443392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0523145710000001, 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055422102000000001, 2024-09-12T07:20:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82753 2024-09-12T07:20:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645016+0000", 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000127847 2024-09-12T07:20:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645024+0000", 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7419999999999997e-06 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645030+0000", 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7229999999999998e-06 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645034+0000", 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9380000000000002e-06 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700132+0000", 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055097699 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700171+0000", 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9093e-05 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700310+0000", 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013905999999999999 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82754 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 446464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0521586640000002, 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055424703999999998, 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82754 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645318+0000", 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027397499999999998 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645322+0000", 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.766e-06 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645331+0000", 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2140000000000002e-06 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645338+0000", 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0929999999999997e-06 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700330+0000", 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054991384999999997 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700369+0000", 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700469+0000", 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9913000000000004e-05 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82755 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 449536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0518536740000002, 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055388918000000002, 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82755 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645500+0000", 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015050999999999999 2024-09-12T07:20:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645502+0000", 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645509+0000", 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3969999999999999e-06 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645513+0000", 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9230000000000004e-06 2024-09-12T07:20:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700507+0000", 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054994692999999997 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700583+0000", 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5656999999999997e-05 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700738+0000", 2024-09-12T07:20:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154869 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82756 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 453632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0516827859999998, 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055451746000000003, 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82756 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645763+0000", 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000243105 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645766+0000", 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0750000000000002e-06 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645774+0000", 2024-09-12T07:20:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9370000000000004e-06 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645783+0000", 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5159999999999994e-06 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700780+0000", 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054997329999999997 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700835+0000", 2024-09-12T07:20:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4789e-05 2024-09-12T07:20:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700972+0000", 2024-09-12T07:20:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013699400000000001 2024-09-12T07:20:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82757 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0514084180000003, 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055382590000000002, 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82757 2024-09-12T07:20:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645926+0000", 2024-09-12T07:20:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131979 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645929+0000", 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0199999999999999e-06 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645935+0000", 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645939+0000", 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.924e-06 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701007+0000", 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067374000000002 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701043+0000", 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6325999999999997e-05 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701177+0000", 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013387300000000001 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82758 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 460800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.051251025, 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055417890999999997, 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82758 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646164+0000", 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000212189 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646167+0000", 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646173+0000", 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4139999999999999e-06 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646177+0000", 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701208+0000", 2024-09-12T07:20:13.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055031322000000001 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701254+0000", 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5701999999999997e-05 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701370+0000", 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115456 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.741 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82759 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 463872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0510184640000002, 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055376838999999997, 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82759 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.742 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646368+0000", 2024-09-12T07:20:13.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183399 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646372+0000", 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9190000000000001e-06 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646380+0000", 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7819999999999999e-06 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646384+0000", 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701407+0000", 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055022763000000002 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701455+0000", 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7660000000000001e-05 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701561+0000", 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010642899999999999 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82760 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0508109149999996, 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055355572999999998, 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.746 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82760 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:13.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646631+0000", 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000238798 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646634+0000", 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.016e-06 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646640+0000", 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2750000000000001e-06 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646644+0000", 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7859999999999998e-06 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701599+0000", 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054955693999999999 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701638+0000", 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8319999999999999e-05 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701747+0000", 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109684 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82761 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0505522860000003, 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055296990999999997, 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82761 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646756+0000", 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105899 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646759+0000", 2024-09-12T07:20:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8380000000000002e-06 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646765+0000", 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646769+0000", 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7900000000000001e-06 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701785+0000", 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055015888999999998 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701832+0000", 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6514000000000002e-05 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701947+0000", 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115833 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82762 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 474112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0504267189999998, 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055346863000000003, 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82762 2024-09-12T07:20:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647063+0000", 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000287115 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647066+0000", 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0189999999999998e-06 2024-09-12T07:20:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647075+0000", 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7099999999999996e-06 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647081+0000", 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6790000000000002e-06 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701977+0000", 2024-09-12T07:20:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054896665999999997 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702015+0000", 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.79e-05 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702123+0000", 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107774 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82763 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.050111802, 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226509999999999, 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82763 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647236+0000", 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000144597 2024-09-12T07:20:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647239+0000", 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1329999999999999e-06 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647245+0000", 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647249+0000", 2024-09-12T07:20:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2270000000000004e-06 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702160+0000", 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054910549000000003 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702205+0000", 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5575000000000003e-05 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702317+0000", 2024-09-12T07:20:13.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011194899999999999 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82764 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 481280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.049946051, 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055229250000000001, 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82764 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647471+0000", 2024-09-12T07:20:13.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000214625 2024-09-12T07:20:13.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647474+0000", 2024-09-12T07:20:13.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8509999999999999e-06 2024-09-12T07:20:13.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647480+0000", 2024-09-12T07:20:13.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.28e-06 2024-09-12T07:20:13.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647484+0000", 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.771e-06 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702346+0000", 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054861551000000001 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702382+0000", 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.65e-05 2024-09-12T07:20:13.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702486+0000", 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010367200000000001 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82765 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 485376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0497105040000001, 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055184729000000002, 2024-09-12T07:20:13.769 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82765 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647681+0000", 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188965 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647693+0000", 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.225e-05 2024-09-12T07:20:13.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647700+0000", 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9929999999999998e-06 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647704+0000", 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9469999999999996e-06 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702522+0000", 2024-09-12T07:20:13.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054817346000000003 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702566+0000", 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4672999999999997e-05 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702677+0000", 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110555 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82766 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 488448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0494912660000004, 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055132717999999997, 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82766 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647959+0000", 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024739599999999999 2024-09-12T07:20:13.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647964+0000", 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9409999999999998e-06 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647975+0000", 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1404e-05 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647983+0000", 2024-09-12T07:20:13.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2969999999999998e-06 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702706+0000", 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054723690999999998 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702743+0000", 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6464999999999997e-05 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702844+0000", 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101524 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82767 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 492544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.049207225, 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055054448999999998, 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82767 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648182+0000", 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018652299999999999 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648185+0000", 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3289999999999998e-06 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648192+0000", 2024-09-12T07:20:13.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7159999999999997e-06 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648196+0000", 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1439999999999996e-06 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702873+0000", 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054676356000000002 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702930+0000", 2024-09-12T07:20:13.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7643999999999999e-05 2024-09-12T07:20:13.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703050+0000", 2024-09-12T07:20:13.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119737 2024-09-12T07:20:13.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.782 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82768 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 495616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.648206+0000", 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0489968420000002, 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055030120000000002, 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82768 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648206+0000", 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648206+0000", 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648400+0000", 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019423500000000001 2024-09-12T07:20:13.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648404+0000", 2024-09-12T07:20:13.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1019999999999999e-06 2024-09-12T07:20:13.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648412+0000", 2024-09-12T07:20:13.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0669999999999992e-06 2024-09-12T07:20:13.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648419+0000", 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2539999999999998e-06 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703088+0000", 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054669199000000002 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703125+0000", 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6847999999999997e-05 2024-09-12T07:20:13.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:13.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703236+0000", 2024-09-12T07:20:13.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000111415 2024-09-12T07:20:13.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.787 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.787 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.787 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:13.789 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:13.790 INFO:teuthology.orchestra.run.smithi046.stdout:1624 writesame 0x288401f thru 0x288f2d1 data_size 0x99(0xb2b3 bytes) 2024-09-12T07:20:13.790 INFO:teuthology.orchestra.run.smithi046.stdout:1626 write 0xd9ded08 thru 0xd9e2b6d (0x3e66 bytes) 2024-09-12T07:20:13.790 INFO:teuthology.orchestra.run.smithi046.stdout:1628 read 0x582292d thru 0x58241ea (0x18be bytes) 2024-09-12T07:20:13.790 INFO:teuthology.orchestra.run.smithi046.stdout:1629 read 0x8df9fbd thru 0x8dfa1a9 (0x1ed bytes) 2024-09-12T07:20:13.790 INFO:teuthology.orchestra.run.smithi046.stdout:1631 writesame 0x7298555 thru 0x729868c data_size 0x2(0x138 bytes) 2024-09-12T07:20:13.790 INFO:teuthology.orchestra.run.smithi046.stdout:1632 punch from 0x434226e to 0x4350a74, (0xe806 bytes) 2024-09-12T07:20:13.791 INFO:teuthology.orchestra.run.smithi046.stdout:1633 writesame 0x5c46cef thru 0x5c4b919 data_size 0x1f(0x4c2b bytes) 2024-09-12T07:20:13.791 INFO:teuthology.orchestra.run.smithi046.stdout:1634 write 0x7283382 thru 0x7284298 (0xf17 bytes) 2024-09-12T07:20:13.791 INFO:teuthology.orchestra.run.smithi046.stdout:1635 read 0x4581fa6 thru 0x4588607 (0x6662 bytes) 2024-09-12T07:20:13.791 INFO:teuthology.orchestra.run.smithi046.stdout:1636 write 0x52287b9 thru 0x522e9ef (0x6237 bytes) 2024-09-12T07:20:13.791 INFO:teuthology.orchestra.run.smithi046.stdout:1637 write 0x3f98fb6 thru 0x3fa8833 (0xf87e bytes) 2024-09-12T07:20:13.791 INFO:teuthology.orchestra.run.smithi046.stdout:1638 read 0x9978cc7 thru 0x997df29 (0x5263 bytes) 2024-09-12T07:20:13.791 INFO:teuthology.orchestra.run.smithi046.stdout:1641 write 0x8be1c57 thru 0x8bf0e9e (0xf248 bytes) 2024-09-12T07:20:13.791 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:13.790+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:13.794 INFO:teuthology.orchestra.run.smithi046.stdout:1642 write 0x66041e4 thru 0x6611454 (0xd271 bytes) 2024-09-12T07:20:13.796 INFO:teuthology.orchestra.run.smithi046.stdout:1644 write 0x54baa38 thru 0x54bdfbb (0x3584 bytes) 2024-09-12T07:20:13.805 INFO:teuthology.orchestra.run.smithi046.stdout:1645 trunc from 0xd9e2b6e to 0x65630e8 2024-09-12T07:20:13.825 INFO:teuthology.orchestra.run.smithi046.stdout:1646 read 0x29b4f41 thru 0x29b8b1c (0x3bdc bytes) 2024-09-12T07:20:13.825 INFO:teuthology.orchestra.run.smithi046.stdout:1648 punch from 0x4c5666d to 0x4c58872, (0x2205 bytes) 2024-09-12T07:20:13.825 INFO:teuthology.orchestra.run.smithi046.stdout:1650 punch from 0x51c30ac to 0x51ca30d, (0x7261 bytes) 2024-09-12T07:20:13.835 INFO:teuthology.orchestra.run.smithi046.stdout:1651 writesame 0x9acb324 thru 0x9ad58a2 data_size 0x1(0xa57f bytes) 2024-09-12T07:20:13.846 INFO:teuthology.orchestra.run.smithi046.stdout:1652 write 0xea33ece thru 0xea43199 (0xf2cc bytes) 2024-09-12T07:20:13.851 INFO:teuthology.orchestra.run.smithi046.stdout:1653 write 0xb98f55 thru 0xba059b (0x7647 bytes) 2024-09-12T07:20:13.867 INFO:teuthology.orchestra.run.smithi046.stdout:1654 punch from 0x6705fbd to 0x670a813, (0x4856 bytes) 2024-09-12T07:20:13.867 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:13.867 INFO:teuthology.orchestra.run.smithi046.stdout:1657 read 0x60b82bb thru 0x60c72ec (0xf032 bytes) 2024-09-12T07:20:13.867 INFO:teuthology.orchestra.run.smithi046.stdout:1658 write 0xa5146df thru 0xa51bc56 (0x7578 bytes) 2024-09-12T07:20:13.871 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:13.871 INFO:teuthology.orchestra.run.smithi046.stdout:1663 read 0x13e91c3 thru 0x13f3358 (0xa196 bytes) 2024-09-12T07:20:13.872 INFO:teuthology.orchestra.run.smithi046.stdout:1664 read 0x7d2a051 thru 0x7d31f25 (0x7ed5 bytes) 2024-09-12T07:20:13.872 INFO:teuthology.orchestra.run.smithi046.stdout:1665 write 0xe568772 thru 0xe56d6e0 (0x4f6f bytes) 2024-09-12T07:20:13.875 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:13.875 INFO:teuthology.orchestra.run.smithi046.stdout:1667 read 0x5bbeae4 thru 0x5bcbeb7 (0xd3d4 bytes) 2024-09-12T07:20:13.876 INFO:teuthology.orchestra.run.smithi046.stdout:1668 read 0x83b21c6 thru 0x83b2640 (0x47b bytes) 2024-09-12T07:20:13.877 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:13.884 INFO:teuthology.orchestra.run.smithi046.stdout:1670 trunc from 0xea4319a to 0x8233c92 2024-09-12T07:20:13.889 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:13.891+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:13.899 INFO:teuthology.orchestra.run.smithi046.stdout:1671 trunc from 0x8233c92 to 0x72e1014 2024-09-12T07:20:13.906 INFO:teuthology.orchestra.run.smithi046.stdout:1674 read 0x593f205 thru 0x594f03d (0xfe39 bytes) 2024-09-12T07:20:13.907 INFO:teuthology.orchestra.run.smithi046.stdout:1675 punch from 0xe93dd3 to 0xe9a4af, (0x66dc bytes) 2024-09-12T07:20:13.921 INFO:teuthology.orchestra.run.smithi046.stdout:1677 write 0xeca1d26 thru 0xecad986 (0xbc61 bytes) 2024-09-12T07:20:13.924 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:13.924 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:13.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.924 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:89520 3.4 3:26c4baf7:::rbd_data.11607a56846.0000000000000031:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3899392~22919 in=22919b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-12T07:20:13.924 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:13.925875+0000", 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.000498505, 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000526627, 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "waiting for sub ops", 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 89520 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.925875+0000", 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.925875+0000", 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.925875+0000", 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5999999999999991e-07 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.925890+0000", 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4219000000000001e-05 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.925904+0000", 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4664000000000001e-05 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.925912+0000", 2024-09-12T07:20:13.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8569999999999995e-06 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.925989+0000", 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.682e-05 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.926129+0000", 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014008799999999999 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:13.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.926277+0000", 2024-09-12T07:20:13.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000147529 2024-09-12T07:20:13.929 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.929 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:13.929 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.929 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:13.929 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:13.929 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 1 2024-09-12T07:20:13.929 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:13.930 INFO:teuthology.orchestra.run.smithi046.stdout:1680 writesame 0xc0239ce thru 0xc03340d data_size 0x8f(0xfa40 bytes) 2024-09-12T07:20:13.930 INFO:teuthology.orchestra.run.smithi046.stdout:1681 write 0xd93e946 thru 0xd940dd1 (0x248c bytes) 2024-09-12T07:20:13.932 INFO:teuthology.orchestra.run.smithi046.stdout:1682 writesame 0xd690845 thru 0xd691290 data_size 0x2(0xa4c bytes) 2024-09-12T07:20:13.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:13.936 INFO:teuthology.orchestra.run.smithi046.stdout:1683 write 0x76de885 thru 0x76e97ba (0xaf36 bytes) 2024-09-12T07:20:13.936 INFO:teuthology.orchestra.run.smithi046.stdout:1685 read 0x2255e20 thru 0x2261cf5 (0xbed6 bytes) 2024-09-12T07:20:13.942 INFO:teuthology.orchestra.run.smithi046.stdout:1686 trunc from 0xecad987 to 0x9cdfa05 2024-09-12T07:20:13.953 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:13.953 INFO:teuthology.orchestra.run.smithi046.stdout:1688 punch from 0x8eacaef to 0x8eb6fb3, (0xa4c4 bytes) 2024-09-12T07:20:13.953 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:13.953 INFO:teuthology.orchestra.run.smithi046.stdout:1690 read 0x707ac19 thru 0x707bfb1 (0x1399 bytes) 2024-09-12T07:20:13.953 INFO:teuthology.orchestra.run.smithi046.stdout:1691 writesame 0x648f1e9 thru 0x64968f2 data_size 0x9(0x770a bytes) 2024-09-12T07:20:13.955 INFO:teuthology.orchestra.run.smithi046.stdout:1692 punch from 0x41c6631 to 0x41d51a7, (0xeb76 bytes) 2024-09-12T07:20:13.965 INFO:teuthology.orchestra.run.smithi046.stdout:1694 trunc from 0x9cdfa05 to 0xcfa428 2024-09-12T07:20:13.991 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:13.992+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:14.023 INFO:teuthology.orchestra.run.smithi046.stdout:1695 write 0xd03a922 thru 0xd040e75 (0x6554 bytes) 2024-09-12T07:20:14.027 INFO:teuthology.orchestra.run.smithi046.stdout:1696 punch from 0x553bc74 to 0x5540f13, (0x529f bytes) 2024-09-12T07:20:14.027 INFO:teuthology.orchestra.run.smithi046.stdout:1697 trunc from 0xd040e76 to 0xd2d633c 2024-09-12T07:20:14.031 INFO:teuthology.orchestra.run.smithi046.stdout:1698 punch from 0x1fbdd8f to 0x1fcc360, (0xe5d1 bytes) 2024-09-12T07:20:14.031 INFO:teuthology.orchestra.run.smithi046.stdout:1699 write 0x610ad77 thru 0x611704f (0xc2d9 bytes) 2024-09-12T07:20:14.045 INFO:teuthology.orchestra.run.smithi046.stdout:1701 write 0xeb1c99c thru 0xeb2b760 (0xedc5 bytes) 2024-09-12T07:20:14.055 INFO:teuthology.orchestra.run.smithi046.stdout:1702 punch from 0x2836e00 to 0x28439b6, (0xcbb6 bytes) 2024-09-12T07:20:14.055 INFO:teuthology.orchestra.run.smithi046.stdout:1703 write 0x5ea12eb thru 0x5ea5ad1 (0x47e7 bytes) 2024-09-12T07:20:14.057 INFO:teuthology.orchestra.run.smithi046.stdout:1704 read 0x26c695e thru 0x26c96a3 (0x2d46 bytes) 2024-09-12T07:20:14.058 INFO:teuthology.orchestra.run.smithi046.stdout:1705 read 0xe429a5d thru 0xe430c2e (0x71d2 bytes) 2024-09-12T07:20:14.058 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.058 INFO:teuthology.orchestra.run.smithi046.stdout:1707 read 0x35d37f9 thru 0x35d7170 (0x3978 bytes) 2024-09-12T07:20:14.058 INFO:teuthology.orchestra.run.smithi046.stdout:1708 write 0xc25f742 thru 0xc26e003 (0xe8c2 bytes) 2024-09-12T07:20:14.061 INFO:teuthology.orchestra.run.smithi046.stdout:1709 read 0xc8da161 thru 0xc8e7358 (0xd1f8 bytes) 2024-09-12T07:20:14.061 INFO:teuthology.orchestra.run.smithi046.stdout:1710 write 0x2fb0f04 thru 0x2fb1874 (0x971 bytes) 2024-09-12T07:20:14.062 INFO:teuthology.orchestra.run.smithi046.stdout:1711 read 0x2bdb0d5 thru 0x2bdff20 (0x4e4c bytes) 2024-09-12T07:20:14.062 INFO:teuthology.orchestra.run.smithi046.stdout:1712 write 0xdcbce8e thru 0xdcc596b (0x8ade bytes) 2024-09-12T07:20:14.066 INFO:teuthology.orchestra.run.smithi046.stdout:1714 write 0xc36ee3f thru 0xc37137d (0x253f bytes) 2024-09-12T07:20:14.069 INFO:teuthology.orchestra.run.smithi046.stdout:1717 read 0xda8ce3a thru 0xda91e60 (0x5027 bytes) 2024-09-12T07:20:14.069 INFO:teuthology.orchestra.run.smithi046.stdout:1719 write 0x82b6e38 thru 0x82c6741 (0xf90a bytes) 2024-09-12T07:20:14.073 INFO:teuthology.orchestra.run.smithi046.stdout:1720 write 0x35ea77 thru 0x367a77 (0x9001 bytes) 2024-09-12T07:20:14.075 INFO:teuthology.orchestra.run.smithi046.stdout:1722 writesame 0xc0d554 thru 0xc1d184 data_size 0x17(0xfc31 bytes) 2024-09-12T07:20:14.077 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:14.077 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:14.079 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.080 INFO:teuthology.orchestra.run.smithi046.stdout:1724 write 0x90aea20 thru 0x90b70dd (0x86be bytes) 2024-09-12T07:20:14.083 INFO:teuthology.orchestra.run.smithi046.stdout:1725 write 0x7ed81b7 thru 0x7edab63 (0x29ad bytes) 2024-09-12T07:20:14.086 INFO:teuthology.orchestra.run.smithi046.stdout:1726 read 0x3c6fb81 thru 0x3c73c96 (0x4116 bytes) 2024-09-12T07:20:14.086 INFO:teuthology.orchestra.run.smithi046.stdout:1730 write 0xa5d8643 thru 0xa5ded13 (0x66d1 bytes) 2024-09-12T07:20:14.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:14.093 INFO:teuthology.orchestra.run.smithi046.stdout:1731 trunc from 0xeb2b761 to 0xb6e0600 2024-09-12T07:20:14.093 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:14.095+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:14.105 INFO:teuthology.orchestra.run.smithi046.stdout:1732 punch from 0x70106d to 0x702513, (0x14a6 bytes) 2024-09-12T07:20:14.105 INFO:teuthology.orchestra.run.smithi046.stdout:1733 punch from 0x83c77f2 to 0x83d6610, (0xee1e bytes) 2024-09-12T07:20:14.105 INFO:teuthology.orchestra.run.smithi046.stdout:1734 read 0x4e5684c thru 0x4e5d6ee (0x6ea3 bytes) 2024-09-12T07:20:14.105 INFO:teuthology.orchestra.run.smithi046.stdout:1735 writesame 0x3c6c55c thru 0x3c78ae4 data_size 0x1(0xc589 bytes) 2024-09-12T07:20:14.123 INFO:teuthology.orchestra.run.smithi046.stdout:1737 writesame 0xae7ccb8 thru 0xae7dec8 data_size 0x5(0x1211 bytes) 2024-09-12T07:20:14.125 INFO:teuthology.orchestra.run.smithi046.stdout:1738 read 0xb21e270 thru 0xb224b5c (0x68ed bytes) 2024-09-12T07:20:14.125 INFO:teuthology.orchestra.run.smithi046.stdout:1739 read 0x501ab80 thru 0x501fa9e (0x4f1f bytes) 2024-09-12T07:20:14.125 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.126 INFO:teuthology.orchestra.run.smithi046.stdout:1742 write 0xac55a18 thru 0xac601bf (0xa7a8 bytes) 2024-09-12T07:20:14.130 INFO:teuthology.orchestra.run.smithi046.stdout:1744 writesame 0x4af02eb thru 0x4af19ec data_size 0x5(0x1702 bytes) 2024-09-12T07:20:14.133 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.133 INFO:teuthology.orchestra.run.smithi046.stdout:1746 read 0x81d2d0 thru 0x8212a6 (0x3fd7 bytes) 2024-09-12T07:20:14.134 INFO:teuthology.orchestra.run.smithi046.stdout:1747 punch from 0x282e86e to 0x2833245, (0x49d7 bytes) 2024-09-12T07:20:14.134 INFO:teuthology.orchestra.run.smithi046.stdout:1748 writesame 0x9a53ed5 thru 0x9a5d430 data_size 0x1(0x955c bytes) 2024-09-12T07:20:14.145 INFO:teuthology.orchestra.run.smithi046.stdout:1749 writesame 0x7a15d91 thru 0x7a19000 data_size 0x18(0x3270 bytes) 2024-09-12T07:20:14.147 INFO:teuthology.orchestra.run.smithi046.stdout:1750 punch from 0x4235325 to 0x423a01a, (0x4cf5 bytes) 2024-09-12T07:20:14.147 INFO:teuthology.orchestra.run.smithi046.stdout:1751 read 0x80b2b23 thru 0x80b80e4 (0x55c2 bytes) 2024-09-12T07:20:14.147 INFO:teuthology.orchestra.run.smithi046.stdout:1752 punch from 0x1239ca4 to 0x123a172, (0x4ce bytes) 2024-09-12T07:20:14.147 INFO:teuthology.orchestra.run.smithi046.stdout:1755 read 0x5efe7eb thru 0x5f0dd21 (0xf537 bytes) 2024-09-12T07:20:14.149 INFO:teuthology.orchestra.run.smithi046.stdout:1756 trunc from 0xb6e0600 to 0xc94a387 2024-09-12T07:20:14.155 INFO:teuthology.orchestra.run.smithi046.stdout:1758 write 0xa66452d thru 0xa66a6b3 (0x6187 bytes) 2024-09-12T07:20:14.164 INFO:teuthology.orchestra.run.smithi046.stdout:1759 trunc from 0xc94a387 to 0x67d6c5b 2024-09-12T07:20:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:1761 write 0xe049ee9 thru 0xe04daf3 (0x3c0b bytes) 2024-09-12T07:20:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:1764 punch from 0xac600a1 to 0xac6a834, (0xa793 bytes) 2024-09-12T07:20:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:1766 punch from 0x50fb9a8 to 0x5101f8c, (0x65e4 bytes) 2024-09-12T07:20:14.193 INFO:teuthology.orchestra.run.smithi046.stdout:1768 read 0x18608fc thru 0x18643e0 (0x3ae5 bytes) 2024-09-12T07:20:14.193 INFO:teuthology.orchestra.run.smithi046.stdout:1769 read 0x725185f thru 0x725b994 (0xa136 bytes) 2024-09-12T07:20:14.193 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:14.194+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:14.198 INFO:teuthology.orchestra.run.smithi046.stdout:1770 write 0xedb216e thru 0xedc00c5 (0xdf58 bytes) 2024-09-12T07:20:14.203 INFO:teuthology.orchestra.run.smithi046.stdout:1772 write 0xd71c56 thru 0xd7ea95 (0xce40 bytes) 2024-09-12T07:20:14.205 INFO:teuthology.orchestra.run.smithi046.stdout:1773 punch from 0x922e0cd to 0x922f284, (0x11b7 bytes) 2024-09-12T07:20:14.205 INFO:teuthology.orchestra.run.smithi046.stdout:1774 writesame 0x3bf1c0a thru 0x3bf3039 data_size 0x8(0x1430 bytes) 2024-09-12T07:20:14.206 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.206 INFO:teuthology.orchestra.run.smithi046.stdout:1776 write 0xeca2aa0 thru 0xecaf1cd (0xc72e bytes) 2024-09-12T07:20:14.209 INFO:teuthology.orchestra.run.smithi046.stdout:1778 flatten 2024-09-12T07:20:14.211 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.2 to 1 (10000) 2024-09-12T07:20:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:20:14.223 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:20:14.225 INFO:teuthology.orchestra.run.smithi046.stdout:1779 trunc from 0xedc00c6 to 0xc5326d4 2024-09-12T07:20:14.227 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:996 3.0 3:05cac305:::rbd_data.116011b0bca6.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.308560+0000", 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.918818708, 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014904776, 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 996 2024-09-12T07:20:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308560+0000", 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308560+0000", 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308561+0000", 2024-09-12T07:20:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.536e-06 2024-09-12T07:20:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308570+0000", 2024-09-12T07:20:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1090000000000004e-06 2024-09-12T07:20:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308584+0000", 2024-09-12T07:20:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3893000000000001e-05 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308592+0000", 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1009999999999992e-06 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308644+0000", 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2117999999999998e-05 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308754+0000", 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109666 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.310773+0000", 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020187310000000002 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.323365+0000", 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012591836 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.323377+0000", 2024-09-12T07:20:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2109999999999999e-05 2024-09-12T07:20:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.323413+0000", 2024-09-12T07:20:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6245999999999998e-05 2024-09-12T07:20:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.323464+0000", 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1430000000000001e-05 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.843990464000001, 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:20:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66600 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.389749+0000", 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.837629721999999, 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032114065999999997, 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66600 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389749+0000", 2024-09-12T07:20:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389749+0000", 2024-09-12T07:20:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389750+0000", 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.421e-06 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389755+0000", 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8250000000000004e-06 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389770+0000", 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5659e-05 2024-09-12T07:20:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389779+0000", 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4440000000000008e-06 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.401597+0000", 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011818265 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.401696+0000", 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9097000000000003e-05 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.402026+0000", 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032977499999999998 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.413052+0000", 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011026094 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421766+0000", 2024-09-12T07:20:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087136069999999999 2024-09-12T07:20:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421822+0000", 2024-09-12T07:20:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6332000000000002e-05 2024-09-12T07:20:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421863+0000", 2024-09-12T07:20:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0547000000000003e-05 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66677 3.0 3:07537075:::rbd_data.1160f022dd61.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.389797+0000", 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.837581204999999, 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022850626999999998, 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66677 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389797+0000", 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389797+0000", 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389798+0000", 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0999999999999997e-07 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389802+0000", 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.817e-06 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389809+0000", 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4900000000000003e-06 2024-09-12T07:20:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389815+0000", 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1099999999999999e-06 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.401560+0000", 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011744409000000001 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.401777+0000", 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000217615 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.402080+0000", 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030243799999999998 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.404470+0000", 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0023900010000000001 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412521+0000", 2024-09-12T07:20:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080516579999999997 2024-09-12T07:20:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412587+0000", 2024-09-12T07:20:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5522e-05 2024-09-12T07:20:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412648+0000", 2024-09-12T07:20:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0757000000000001e-05 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.7809122, 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:20:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:20:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.777207115, 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:20:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:20:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:20:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:20:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:20:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:20:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:20:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:20:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:20:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:69559 3.1a 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.681624+0000", 2024-09-12T07:20:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.545754193, 2024-09-12T07:20:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023561162, 2024-09-12T07:20:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 69559 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.681624+0000", 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.681624+0000", 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.681626+0000", 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5409999999999999e-06 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683088+0000", 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001462009 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683122+0000", 2024-09-12T07:20:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-12T07:20:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683133+0000", 2024-09-12T07:20:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1559000000000001e-05 2024-09-12T07:20:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683225+0000", 2024-09-12T07:20:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2330000000000005e-05 2024-09-12T07:20:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683325+0000", 2024-09-12T07:20:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9513999999999996e-05 2024-09-12T07:20:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683782+0000", 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00045661299999999997 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.691512+0000", 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0077308530000000002 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.705112+0000", 2024-09-12T07:20:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013599862000000001 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.705155+0000", 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2988999999999998e-05 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.705185+0000", 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.995e-05 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:69560 3.1a 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.683195+0000", 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.544182954, 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024920327999999999, 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 69560 2024-09-12T07:20:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683195+0000", 2024-09-12T07:20:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683195+0000", 2024-09-12T07:20:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683197+0000", 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.624e-06 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683202+0000", 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1189999999999996e-06 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683892+0000", 2024-09-12T07:20:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00068997999999999998 2024-09-12T07:20:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683905+0000", 2024-09-12T07:20:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3066e-05 2024-09-12T07:20:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.687754+0000", 2024-09-12T07:20:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038485139999999999 2024-09-12T07:20:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.687860+0000", 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106712 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.688125+0000", 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026430699999999998 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.692505+0000", 2024-09-12T07:20:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0043800030000000004 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.708032+0000", 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01552693 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.708079+0000", 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7775999999999998e-05 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.708116+0000", 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6297000000000002e-05 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:69639 3.1a 3:5ec4a676:::rbd_data.1160f022dd61.0000000000000019:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.685937+0000", 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.541441667000001, 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022266522, 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 69639 2024-09-12T07:20:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685937+0000", 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685937+0000", 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685938+0000", 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02e-06 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685951+0000", 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3621000000000001e-05 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685961+0000", 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0210999999999999e-05 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685968+0000", 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9040000000000003e-06 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.691551+0000", 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055829089999999996 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.691668+0000", 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117186 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.708203+0000", 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016534671000000001 2024-09-12T07:20:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77979 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 8355840~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.527404+0000", 2024-09-12T07:20:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6999738779999998, 2024-09-12T07:20:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011196462000000001, 2024-09-12T07:20:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77979 2024-09-12T07:20:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.527404+0000", 2024-09-12T07:20:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.527404+0000", 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.527405+0000", 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1300000000000001e-07 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.527407+0000", 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9410000000000001e-06 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.527413+0000", 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9359999999999996e-06 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.527417+0000", 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3800000000000004e-06 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.538300+0000", 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010883345000000001 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.538383+0000", 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2657999999999994e-05 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.538601+0000", 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000217789 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78042 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:b [read 8323072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528554+0000", 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6988238960000004, 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011297491, 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78042 2024-09-12T07:20:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528554+0000", 2024-09-12T07:20:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528554+0000", 2024-09-12T07:20:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528555+0000", 2024-09-12T07:20:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4200000000000001e-07 2024-09-12T07:20:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528557+0000", 2024-09-12T07:20:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0480000000000001e-06 2024-09-12T07:20:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528565+0000", 2024-09-12T07:20:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2490000000000001e-06 2024-09-12T07:20:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528578+0000", 2024-09-12T07:20:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2513e-05 2024-09-12T07:20:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539560+0000", 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010982228 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539642+0000", 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1780000000000006e-05 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539852+0000", 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021023099999999999 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78043 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 7798784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528586+0000", 2024-09-12T07:20:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6987926509999998, 2024-09-12T07:20:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011197245999999999, 2024-09-12T07:20:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78043 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528586+0000", 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528586+0000", 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528586+0000", 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9200000000000001e-07 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528588+0000", 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2709999999999999e-06 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528597+0000", 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4130000000000006e-06 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528602+0000", 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2950000000000001e-06 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539546+0000", 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010943575000000001 2024-09-12T07:20:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539626+0000", 2024-09-12T07:20:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0531999999999994e-05 2024-09-12T07:20:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539783+0000", 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000156668 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78044 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 7929856~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528609+0000", 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6987687900000008, 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011471324, 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78044 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528609+0000", 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528609+0000", 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528610+0000", 2024-09-12T07:20:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2199999999999999e-07 2024-09-12T07:20:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528612+0000", 2024-09-12T07:20:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.888e-06 2024-09-12T07:20:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528620+0000", 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3140000000000004e-06 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528625+0000", 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.87e-06 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539809+0000", 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011183659 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539865+0000", 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6563e-05 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540081+0000", 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000215608 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78045 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 8060928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528631+0000", 2024-09-12T07:20:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6987468910000008, 2024-09-12T07:20:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011695537000000001, 2024-09-12T07:20:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78045 2024-09-12T07:20:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528631+0000", 2024-09-12T07:20:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528631+0000", 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528632+0000", 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.46e-07 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528634+0000", 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9700000000000002e-06 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528640+0000", 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7329999999999997e-06 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528646+0000", 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1699999999999996e-06 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540111+0000", 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011465444999999999 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540164+0000", 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2927999999999999e-05 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540327+0000", 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016294500000000001 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78046 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 8192000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528655+0000", 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6987232599999995, 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012006785000000001, 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78046 2024-09-12T07:20:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528655+0000", 2024-09-12T07:20:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528655+0000", 2024-09-12T07:20:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528655+0000", 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9999999999999998e-07 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528657+0000", 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.125e-06 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528664+0000", 2024-09-12T07:20:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5660000000000003e-06 2024-09-12T07:20:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528668+0000", 2024-09-12T07:20:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2570000000000001e-06 2024-09-12T07:20:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540386+0000", 2024-09-12T07:20:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011717627 2024-09-12T07:20:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540470+0000", 2024-09-12T07:20:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3891e-05 2024-09-12T07:20:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540662+0000", 2024-09-12T07:20:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000191919 2024-09-12T07:20:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78047 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 8323072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528677+0000", 2024-09-12T07:20:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6987012319999995, 2024-09-12T07:20:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012383072, 2024-09-12T07:20:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78047 2024-09-12T07:20:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528677+0000", 2024-09-12T07:20:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528677+0000", 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528677+0000", 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.65e-07 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528680+0000", 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4449999999999999e-06 2024-09-12T07:20:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528687+0000", 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5329999999999997e-06 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528692+0000", 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540693+0000", 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012000613 2024-09-12T07:20:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540781+0000", 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8178999999999997e-05 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.541060+0000", 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027922400000000001 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78287 3.19 3:99bc29a5:::rbd_data.116087d6d6a2.0000000000000047:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.635267+0000", 2024-09-12T07:20:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5921108529999994, 2024-09-12T07:20:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022254682000000001, 2024-09-12T07:20:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78287 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635267+0000", 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635267+0000", 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635269+0000", 2024-09-12T07:20:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.178e-06 2024-09-12T07:20:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635273+0000", 2024-09-12T07:20:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.121e-06 2024-09-12T07:20:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635281+0000", 2024-09-12T07:20:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5779999999999999e-06 2024-09-12T07:20:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635286+0000", 2024-09-12T07:20:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8199999999999996e-06 2024-09-12T07:20:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635352+0000", 2024-09-12T07:20:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5571e-05 2024-09-12T07:20:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635473+0000", 2024-09-12T07:20:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000121795 2024-09-12T07:20:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.639340+0000", 2024-09-12T07:20:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038665679999999999 2024-09-12T07:20:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.657349+0000", 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018008941000000001 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.657402+0000", 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3142e-05 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.657466+0000", 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4325000000000005e-05 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.657522+0000", 2024-09-12T07:20:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5643000000000002e-05 2024-09-12T07:20:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78288 3.1a 3:58d7858f:::rbd_data.116087d6d6a2.0000000000000048:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.635707+0000", 2024-09-12T07:20:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5916709630000003, 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016418782, 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78288 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635707+0000", 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635707+0000", 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635708+0000", 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8100000000000002e-07 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635712+0000", 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635721+0000", 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6539999999999994e-06 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635726+0000", 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5580000000000002e-06 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635768+0000", 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1897999999999998e-05 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635844+0000", 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6446999999999995e-05 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.638143+0000", 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022985079999999999 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.651990+0000", 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013846712000000001 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.652009+0000", 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9548e-05 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.652070+0000", 2024-09-12T07:20:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0523999999999999e-05 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.652126+0000", 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6462000000000003e-05 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.7788608359999998, 2024-09-12T07:20:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:20:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:20:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:20:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:20:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:20:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:20:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:20:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:20:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:88881 3.1a 3:5dc3e11f:::rbd_data.11607a56846.0000000000000016:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 2129920~15013] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:13.612488+0000", 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.61488987699999997, 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0163359, 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 88881 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.612488+0000", 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.612488+0000", 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.612492+0000", 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7139999999999999e-06 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616402+0000", 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039097059999999998 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616427+0000", 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5667999999999999e-05 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616440+0000", 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2495e-05 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616525+0000", 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5442e-05 2024-09-12T07:20:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616693+0000", 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000167691 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.617037+0000", 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034386099999999999 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.627285+0000", 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010248554999999999 2024-09-12T07:20:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.628694+0000", 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001408379 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.628766+0000", 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1766000000000004e-05 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.628824+0000", 2024-09-12T07:20:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8622999999999998e-05 2024-09-12T07:20:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:14.341 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:14.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-12T07:20:14.343 INFO:teuthology.orchestra.run.smithi046.stdout:1781 write 0x4e7caf9 thru 0x4e86f5c (0xa464 bytes) 2024-09-12T07:20:14.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:14.346 INFO:teuthology.orchestra.run.smithi046.stdout:1782 trunc from 0xc5326d4 to 0x880821e 2024-09-12T07:20:14.346 INFO:teuthology.orchestra.run.smithi046.stdout:1784 writesame 0x392878 thru 0x39af8d data_size 0x1(0x8716 bytes) 2024-09-12T07:20:14.346 INFO:teuthology.orchestra.run.smithi046.stdout:1787 read 0x58db5a9 thru 0x58e8973 (0xd3cb bytes) 2024-09-12T07:20:14.346 INFO:teuthology.orchestra.run.smithi046.stdout:1789 writesame 0x167433c thru 0x1681216 data_size 0x59(0xcedb bytes) 2024-09-12T07:20:14.346 INFO:teuthology.orchestra.run.smithi046.stdout:1790 trunc from 0x880821e to 0xc28488b 2024-09-12T07:20:14.346 INFO:teuthology.orchestra.run.smithi046.stdout:1791 writesame 0xdaf7767 thru 0xdafe535 data_size 0x1(0x6dcf bytes) 2024-09-12T07:20:14.346 INFO:teuthology.orchestra.run.smithi046.stdout:1795 write 0x6c06062 thru 0x6c06ad2 (0xa71 bytes) 2024-09-12T07:20:14.346 INFO:teuthology.orchestra.run.smithi046.stdout:1797 trunc from 0xdafe536 to 0x5a5d659 2024-09-12T07:20:14.346 INFO:teuthology.orchestra.run.smithi046.stdout:1798 punch from 0x90499a to 0x905e4c, (0x14b2 bytes) 2024-09-12T07:20:14.347 INFO:teuthology.orchestra.run.smithi046.stdout:1799 trunc from 0x5a5d659 to 0x1ff9629 2024-09-12T07:20:14.347 INFO:teuthology.orchestra.run.smithi046.stdout:1800 punch from 0x19957db to 0x1996d0b, (0x1530 bytes) 2024-09-12T07:20:14.347 INFO:teuthology.orchestra.run.smithi046.stdout:1801 write 0x1bd2152 thru 0x1bd972b (0x75da bytes) 2024-09-12T07:20:14.347 INFO:teuthology.orchestra.run.smithi046.stdout:1802 write 0x229765a thru 0x22a52ff (0xdca6 bytes) 2024-09-12T07:20:14.347 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:14.344+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:14.351 INFO:teuthology.orchestra.run.smithi046.stdout:1803 writesame 0xba3b4e1 thru 0xba4aaa8 data_size 0xd(0xf5c8 bytes) 2024-09-12T07:20:14.351 INFO:teuthology.orchestra.run.smithi046.stdout:1804 writesame 0x83c1551 thru 0x83cdfa5 data_size 0x1(0xca55 bytes) 2024-09-12T07:20:14.365 INFO:teuthology.orchestra.run.smithi046.stdout:1805 read 0x2259ab8 thru 0x226183c (0x7d85 bytes) 2024-09-12T07:20:14.366 INFO:teuthology.orchestra.run.smithi046.stdout:1807 write 0x2699ea3 thru 0x26a7c85 (0xdde3 bytes) 2024-09-12T07:20:14.368 INFO:teuthology.orchestra.run.smithi046.stdout:1811 read 0x64f9e75 thru 0x64fe064 (0x41f0 bytes) 2024-09-12T07:20:14.368 INFO:teuthology.orchestra.run.smithi046.stdout:1812 writesame 0xff713f thru 0x1000db4 data_size 0x7(0x9c76 bytes) 2024-09-12T07:20:14.371 INFO:teuthology.orchestra.run.smithi046.stdout:1813 writesame 0xa705d3b thru 0xa7074ce data_size 0x6(0x1794 bytes) 2024-09-12T07:20:14.380 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:14.380 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:14.382 INFO:teuthology.orchestra.run.smithi046.stdout:1814 trunc from 0xba4aaa9 to 0x6654990 2024-09-12T07:20:14.403 INFO:teuthology.orchestra.run.smithi046.stdout:1815 writesame 0x72edb51 thru 0x72f7f54 data_size 0x6(0xa404 bytes) 2024-09-12T07:20:14.404 INFO:teuthology.orchestra.run.smithi046.stdout:1820 read 0x9a4e1c thru 0x9a70ae (0x2293 bytes) 2024-09-12T07:20:14.405 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:14.405 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:14.416 INFO:teuthology.orchestra.run.smithi046.stdout:1821 writesame 0xa0c887c thru 0xa0cff88 data_size 0x1(0x770d bytes) 2024-09-12T07:20:14.416 INFO:teuthology.orchestra.run.smithi046.stdout:1822 punch from 0x6dd6dcc to 0x6ddc7e3, (0x5a17 bytes) 2024-09-12T07:20:14.416 INFO:teuthology.orchestra.run.smithi046.stdout:1823 write 0x2abc3f0 thru 0x2ac37b3 (0x73c4 bytes) 2024-09-12T07:20:14.417 INFO:teuthology.orchestra.run.smithi046.stdout:1824 trunc from 0xa0cff89 to 0x9f77af7 2024-09-12T07:20:14.422 INFO:teuthology.orchestra.run.smithi046.stdout:1826 write 0x1870276 thru 0x1873c1c (0x39a7 bytes) 2024-09-12T07:20:14.423 INFO:teuthology.orchestra.run.smithi046.stdout:1828 read 0x7663eb8 thru 0x76642d2 (0x41b bytes) 2024-09-12T07:20:14.423 INFO:teuthology.orchestra.run.smithi046.stdout:1829 write 0x7d5f341 thru 0x7d6f2d0 (0xff90 bytes) 2024-09-12T07:20:14.426 INFO:teuthology.orchestra.run.smithi046.stdout:1834 read 0x649c596 thru 0x649ccb9 (0x724 bytes) 2024-09-12T07:20:14.427 INFO:teuthology.orchestra.run.smithi046.stdout:1835 punch from 0x335bf2a to 0x335c85a, (0x930 bytes) 2024-09-12T07:20:14.434 INFO:teuthology.orchestra.run.smithi046.stdout:1836 trunc from 0x9f77af7 to 0x3a81994 2024-09-12T07:20:14.439 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:14.439 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:14.444 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:14.445+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:14.447 INFO:teuthology.orchestra.run.smithi046.stdout:1839 read 0x27168ed thru 0x2719d68 (0x347c bytes) 2024-09-12T07:20:14.447 INFO:teuthology.orchestra.run.smithi046.stdout:1840 punch from 0x2685477 to 0x26933cc, (0xdf55 bytes) 2024-09-12T07:20:14.460 INFO:teuthology.orchestra.run.smithi046.stdout:1842 write 0x9f58ac0 thru 0x9f61464 (0x89a5 bytes) 2024-09-12T07:20:14.467 INFO:teuthology.orchestra.run.smithi046.stdout:1844 punch from 0x9358847 to 0x93637f1, (0xafaa bytes) 2024-09-12T07:20:14.467 INFO:teuthology.orchestra.run.smithi046.stdout:1846 read 0x9db1b5e thru 0x9dbb429 (0x98cc bytes) 2024-09-12T07:20:14.467 INFO:teuthology.orchestra.run.smithi046.stdout:1847 write 0x47ee50b thru 0x47f27a8 (0x429e bytes) 2024-09-12T07:20:14.473 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:14.480 INFO:teuthology.orchestra.run.smithi046.stdout:1848 write 0xaca84f2 thru 0xacb44c1 (0xbfd0 bytes) 2024-09-12T07:20:14.484 INFO:teuthology.orchestra.run.smithi046.stdout:1849 read 0x5da52b6 thru 0x5daf31f (0xa06a bytes) 2024-09-12T07:20:14.485 INFO:teuthology.orchestra.run.smithi046.stdout:1850 write 0x4661496 thru 0x466ea14 (0xd57f bytes) 2024-09-12T07:20:14.488 INFO:teuthology.orchestra.run.smithi046.stdout:1851 write 0x798b3f9 thru 0x798c100 (0xd08 bytes) 2024-09-12T07:20:14.489 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:14.489 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:14.493 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:14.494 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:14.496 INFO:teuthology.orchestra.run.smithi046.stdout:1852 writesame 0xad880a0 thru 0xad97b53 data_size 0xa(0xfab4 bytes) 2024-09-12T07:20:14.497 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.497 INFO:teuthology.orchestra.run.smithi046.stdout:1854 punch from 0x1411ebe to 0x1421864, (0xf9a6 bytes) 2024-09-12T07:20:14.497 INFO:teuthology.orchestra.run.smithi046.stdout:1856 write 0x20354f2 thru 0x2043351 (0xde60 bytes) 2024-09-12T07:20:14.498 INFO:teuthology.orchestra.run.smithi046.stdout:1857 punch from 0x78bc584 to 0x78bffca, (0x3a46 bytes) 2024-09-12T07:20:14.498 INFO:teuthology.orchestra.run.smithi046.stdout:1858 read 0x629971d thru 0x62a4575 (0xae59 bytes) 2024-09-12T07:20:14.499 INFO:teuthology.orchestra.run.smithi046.stdout:1859 write 0x8f8d985 thru 0x8f9b7e8 (0xde64 bytes) 2024-09-12T07:20:14.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:14.505 INFO:teuthology.orchestra.run.smithi046.stdout:1862 write 0x96db487 thru 0x96e28df (0x7459 bytes) 2024-09-12T07:20:14.512 INFO:teuthology.orchestra.run.smithi046.stdout:1865 read 0xa4242f3 thru 0xa425244 (0xf52 bytes) 2024-09-12T07:20:14.518 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:14.520 INFO:teuthology.orchestra.run.smithi046.stdout:1867 write 0xbbfb6b0 thru 0xbbfea34 (0x3385 bytes) 2024-09-12T07:20:14.522 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.522 INFO:teuthology.orchestra.run.smithi046.stdout:1870 read 0xa53abc8 thru 0xa544dd1 (0xa20a bytes) 2024-09-12T07:20:14.522 INFO:teuthology.orchestra.run.smithi046.stdout:1873 write 0x268c0e3 thru 0x269b828 (0xf746 bytes) 2024-09-12T07:20:14.524 INFO:teuthology.orchestra.run.smithi046.stdout:1874 writesame 0xa82c75d thru 0xa83733f data_size 0x1(0xabe3 bytes) 2024-09-12T07:20:14.538 INFO:teuthology.orchestra.run.smithi046.stdout:1875 write 0x43e9a12 thru 0x43f2e46 (0x9435 bytes) 2024-09-12T07:20:14.542 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:14.544 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:14.546+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:14.554 INFO:teuthology.orchestra.run.smithi046.stdout:1876 trunc from 0xbbfea35 to 0x3393c4a 2024-09-12T07:20:14.561 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:14.562 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:14.571 INFO:teuthology.orchestra.run.smithi046.stdout:1877 read 0x3196438 thru 0x319fae6 (0x96af bytes) 2024-09-12T07:20:14.571 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.583 INFO:teuthology.orchestra.run.smithi046.stdout:1882 writesame 0x8921374 thru 0x892deb2 data_size 0x1(0xcb3f bytes) 2024-09-12T07:20:14.583 INFO:teuthology.orchestra.run.smithi046.stdout:1883 read 0x53a557e thru 0x53aefa7 (0x9a2a bytes) 2024-09-12T07:20:14.584 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.584 INFO:teuthology.orchestra.run.smithi046.stdout:1886 write 0x68a4c0e thru 0x68b009a (0xb48d bytes) 2024-09-12T07:20:14.588 INFO:teuthology.orchestra.run.smithi046.stdout:1887 read 0x569e992 thru 0x56ac442 (0xdab1 bytes) 2024-09-12T07:20:14.588 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.588 INFO:teuthology.orchestra.run.smithi046.stdout:1891 write 0x717c915 thru 0x7182b86 (0x6272 bytes) 2024-09-12T07:20:14.592 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.592 INFO:teuthology.orchestra.run.smithi046.stdout:1895 read 0x47f9eef thru 0x48008a6 (0x69b8 bytes) 2024-09-12T07:20:14.592 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.592 INFO:teuthology.orchestra.run.smithi046.stdout:1899 punch from 0x407b0d7 to 0x408622b, (0xb154 bytes) 2024-09-12T07:20:14.599 INFO:teuthology.orchestra.run.smithi046.stdout:1900 trunc from 0x892deb3 to 0x2619e2c 2024-09-12T07:20:14.621 INFO:teuthology.orchestra.run.smithi046.stdout:1901 write 0x283c444 thru 0x28422f1 (0x5eae bytes) 2024-09-12T07:20:14.640 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:14.640 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:14.642 INFO:teuthology.orchestra.run.smithi046.stdout:1902 writesame 0xe80c23a thru 0xe80ea6d data_size 0x4(0x2834 bytes) 2024-09-12T07:20:14.642 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.643 INFO:teuthology.orchestra.run.smithi046.stdout:1905 read 0x8ee67a4 thru 0x8eecc76 (0x64d3 bytes) 2024-09-12T07:20:14.643 INFO:teuthology.orchestra.run.smithi046.stdout:1906 write 0xa3a58d6 thru 0xa3a5a5a (0x185 bytes) 2024-09-12T07:20:14.645 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:14.646+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:14.645 INFO:teuthology.orchestra.run.smithi046.stdout:1908 read 0x5fded3f thru 0x5fee3d0 (0xf692 bytes) 2024-09-12T07:20:14.645 INFO:teuthology.orchestra.run.smithi046.stdout:1909 write 0xc416821 thru 0xc4175a5 (0xd85 bytes) 2024-09-12T07:20:14.648 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:14.661 INFO:teuthology.orchestra.run.smithi046.stdout:1912 trunc from 0xe80ea6e to 0x35b5d77 2024-09-12T07:20:14.672 INFO:teuthology.orchestra.run.smithi046.stdout:1913 punch from 0x1d934fd to 0x1da0412, (0xcf15 bytes) 2024-09-12T07:20:14.672 INFO:teuthology.orchestra.run.smithi046.stdout:1914 punch from 0x4f96b1 to 0x507733, (0xe082 bytes) 2024-09-12T07:20:14.689 INFO:teuthology.orchestra.run.smithi046.stdout:1916 write 0xd4c822b thru 0xd4caf4d (0x2d23 bytes) 2024-09-12T07:20:14.692 INFO:teuthology.orchestra.run.smithi046.stdout:1917 read 0xb596ea8 thru 0xb5a37c2 (0xc91b bytes) 2024-09-12T07:20:14.692 INFO:teuthology.orchestra.run.smithi046.stdout:1918 read 0x65ce97 thru 0x668f15 (0xc07f bytes) 2024-09-12T07:20:14.693 INFO:teuthology.orchestra.run.smithi046.stdout:1919 read 0x9faa9e1 thru 0x9fb8d27 (0xe347 bytes) 2024-09-12T07:20:14.693 INFO:teuthology.orchestra.run.smithi046.stdout:1920 writesame 0xd12836d thru 0xd12e6db data_size 0x1(0x636f bytes) 2024-09-12T07:20:14.707 INFO:teuthology.orchestra.run.smithi046.stdout:1921 writesame 0x5352292 thru 0x535c9c9 data_size 0x4(0xa738 bytes) 2024-09-12T07:20:14.714 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.714 INFO:teuthology.orchestra.run.smithi046.stdout:1923 write 0x17194ea thru 0x172461f (0xb136 bytes) 2024-09-12T07:20:14.715 INFO:teuthology.orchestra.run.smithi046.stdout:1925 read 0x9f1c821 thru 0x9f2a621 (0xde01 bytes) 2024-09-12T07:20:14.716 INFO:teuthology.orchestra.run.smithi046.stdout:1926 read 0xd401205 thru 0xd4102ce (0xf0ca bytes) 2024-09-12T07:20:14.716 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.716 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.716 INFO:teuthology.orchestra.run.smithi046.stdout:1929 write 0x285100e thru 0x285b31f (0xa312 bytes) 2024-09-12T07:20:14.717 INFO:teuthology.orchestra.run.smithi046.stdout:1930 punch from 0x2b6aa37 to 0x2b786dc, (0xdca5 bytes) 2024-09-12T07:20:14.718 INFO:teuthology.orchestra.run.smithi046.stdout:1931 punch from 0x78bbf2e to 0x78bd259, (0x132b bytes) 2024-09-12T07:20:14.718 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.718 INFO:teuthology.orchestra.run.smithi046.stdout:1933 write 0x3b7c350 thru 0x3b8422f (0x7ee0 bytes) 2024-09-12T07:20:14.722 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:14.722 INFO:teuthology.orchestra.run.smithi046.stdout:1935 read 0x77c77d3 thru 0x77d5e96 (0xe6c4 bytes) 2024-09-12T07:20:14.722 INFO:teuthology.orchestra.run.smithi046.stdout:1936 read 0xcf1d152 thru 0xcf2715d (0xa00c bytes) 2024-09-12T07:20:14.722 INFO:teuthology.orchestra.run.smithi046.stdout:1937 write 0xa1ca1fa thru 0xa1cf1dc (0x4fe3 bytes) 2024-09-12T07:20:14.731 INFO:teuthology.orchestra.run.smithi046.stdout:1938 trunc from 0xd4caf4e to 0x9106d2a 2024-09-12T07:20:14.746 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:14.748+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:14.751 INFO:teuthology.orchestra.run.smithi046.stdout:1939 write 0xc2a85bb thru 0xc2af766 (0x71ac bytes) 2024-09-12T07:20:14.754 INFO:teuthology.orchestra.run.smithi046.stdout:1940 read 0x2da56af thru 0x2db136e (0xbcc0 bytes) 2024-09-12T07:20:14.754 INFO:teuthology.orchestra.run.smithi046.stdout:1941 read 0x1059825 thru 0x1061f7d (0x8759 bytes) 2024-09-12T07:20:14.755 INFO:teuthology.orchestra.run.smithi046.stdout:1942 writesame 0x43539ae thru 0x4358457 data_size 0x2(0x4aaa bytes) 2024-09-12T07:20:14.762 INFO:teuthology.orchestra.run.smithi046.stdout:1943 writesame 0x916760d thru 0x916e720 data_size 0x1(0x7114 bytes) 2024-09-12T07:20:14.778 INFO:teuthology.orchestra.run.smithi046.stdout:1945 writesame 0x481269e thru 0x4815f0c data_size 0x1(0x386f bytes) 2024-09-12T07:20:14.785 INFO:teuthology.orchestra.run.smithi046.stdout:1946 clone 15 order 22 su 16384 sc 6 2024-09-12T07:20:14.787 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:14.787 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:14.796 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:14.880 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:14.882+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:14.945 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:14.945 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:14.948 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:14.949+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:14.953 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:15.048 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:15.050+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:15.101 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:15.101 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:15.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:15.150 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:15.152+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:15.216 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.4 to 1 (10000) 2024-09-12T07:20:15.232 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-12T07:20:15.250 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:15.252+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:15.269 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:15.269 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:15.281 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:15.351 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:15.352+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:15.399 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:15.399 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:15.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:15.413 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-12T07:20:15.452 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:15.454+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:15.481 INFO:teuthology.orchestra.run.smithi046.stdout:1947 read 0xade87f2 thru 0xadf238c (0x9b9b bytes) 2024-09-12T07:20:15.484 INFO:teuthology.orchestra.run.smithi046.stdout:1948 trunc from 0xc2af767 to 0xa9e8add 2024-09-12T07:20:15.523 INFO:teuthology.orchestra.run.smithi046.stdout:1950 read 0x9ffc145 thru 0xa007788 (0xb644 bytes) 2024-09-12T07:20:15.523 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:15.523 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:15.524 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:15.524 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:15.531 INFO:teuthology.orchestra.run.smithi046.stdout:1954 write 0xd652cda thru 0xd660ee3 (0xe20a bytes) 2024-09-12T07:20:15.533 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:15.538 INFO:teuthology.orchestra.run.smithi046.stdout:1955 write 0x4e95948 thru 0x4e9a07c (0x4735 bytes) 2024-09-12T07:20:15.553 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:15.554+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:15.560 INFO:teuthology.orchestra.run.smithi046.stdout:1956 trunc from 0xd660ee4 to 0xe89a389 2024-09-12T07:20:15.565 INFO:teuthology.orchestra.run.smithi046.stdout:1958 writesame 0xfd6a72 thru 0xfd9e2b data_size 0x3(0x33ba bytes) 2024-09-12T07:20:15.623 INFO:teuthology.orchestra.run.smithi046.stdout:1960 read 0xb77b288 thru 0xb78a64f (0xf3c8 bytes) 2024-09-12T07:20:15.634 INFO:teuthology.orchestra.run.smithi046.stdout:1961 trunc from 0xe89a389 to 0x56da609 2024-09-12T07:20:15.654 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:15.656+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:15.676 INFO:teuthology.orchestra.run.smithi046.stdout:1962 read 0x26d4a1a thru 0x26de021 (0x9608 bytes) 2024-09-12T07:20:15.677 INFO:teuthology.orchestra.run.smithi046.stdout:1963 read 0x158d3e0 thru 0x158d7cd (0x3ee bytes) 2024-09-12T07:20:15.677 INFO:teuthology.orchestra.run.smithi046.stdout:1965 read 0x299c92b thru 0x29ab3d8 (0xeaae bytes) 2024-09-12T07:20:15.693 INFO:teuthology.orchestra.run.smithi046.stdout:1966 write 0xd47a535 thru 0xd4873fc (0xcec8 bytes) 2024-09-12T07:20:15.696 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:15.696 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:15.698 INFO:teuthology.orchestra.run.smithi046.stdout:1967 read 0x5a0c4fd thru 0x5a12eb3 (0x69b7 bytes) 2024-09-12T07:20:15.699 INFO:teuthology.orchestra.run.smithi046.stdout:1968 read 0x9b61b9e thru 0x9b64cd4 (0x3137 bytes) 2024-09-12T07:20:15.702 INFO:teuthology.orchestra.run.smithi046.stdout:1969 trunc from 0xd4873fd to 0x9e2c211 2024-09-12T07:20:15.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:15.722 INFO:teuthology.orchestra.run.smithi046.stdout:1970 writesame 0xe05cf6d thru 0xe06ca94 data_size 0x4(0xfb28 bytes) 2024-09-12T07:20:15.722 INFO:teuthology.orchestra.run.smithi046.stdout:1972 write 0x16ce640 thru 0x16d84f2 (0x9eb3 bytes) 2024-09-12T07:20:15.756 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:15.757+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:15.803 INFO:teuthology.orchestra.run.smithi046.stdout:1973 trunc from 0xe06ca95 to 0xdc7e531 2024-09-12T07:20:15.808 INFO:teuthology.orchestra.run.smithi046.stdout:1976 writesame 0x76a7e22 thru 0x76b2f40 data_size 0x1(0xb11f bytes) 2024-09-12T07:20:15.824 INFO:teuthology.orchestra.run.smithi046.stdout:1977 write 0xb045fd5 thru 0xb04ee62 (0x8e8e bytes) 2024-09-12T07:20:15.829 INFO:teuthology.orchestra.run.smithi046.stdout:1978 writesame 0xa8b4a9d thru 0xa8bb98b data_size 0x1(0x6eef bytes) 2024-09-12T07:20:15.839 INFO:teuthology.orchestra.run.smithi046.stdout:1979 write 0x1245060 thru 0x125055f (0xb500 bytes) 2024-09-12T07:20:15.844 INFO:teuthology.orchestra.run.smithi046.stdout:1980 write 0x5bb1a3b thru 0x5bb9071 (0x7637 bytes) 2024-09-12T07:20:15.849 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:15.849 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:15.852 INFO:teuthology.orchestra.run.smithi046.stdout:1983 trunc from 0xdc7e531 to 0x8ae5f78 2024-09-12T07:20:15.856 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:15.858+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:15.859 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:15.861 INFO:teuthology.orchestra.run.smithi046.stdout:1984 punch from 0x8aabf45 to 0x8ab62f8, (0xa3b3 bytes) 2024-09-12T07:20:15.861 INFO:teuthology.orchestra.run.smithi046.stdout:1986 writesame 0x66b13ed thru 0x66b2d10 data_size 0x2(0x1924 bytes) 2024-09-12T07:20:15.865 INFO:teuthology.orchestra.run.smithi046.stdout:1987 read 0x4f912b thru 0x50278c (0x9662 bytes) 2024-09-12T07:20:15.866 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:15.866 INFO:teuthology.orchestra.run.smithi046.stdout:1991 read 0xec01b9 thru 0xec2b7b (0x29c3 bytes) 2024-09-12T07:20:15.866 INFO:teuthology.orchestra.run.smithi046.stdout:1992 read 0x78fd22b thru 0x790208f (0x4e65 bytes) 2024-09-12T07:20:15.873 INFO:teuthology.orchestra.run.smithi046.stdout:1994 trunc from 0x8ae5f78 to 0x329f529 2024-09-12T07:20:15.928 INFO:teuthology.orchestra.run.smithi046.stdout:1995 punch from 0x17e6512 to 0x17f21f2, (0xbce0 bytes) 2024-09-12T07:20:15.928 INFO:teuthology.orchestra.run.smithi046.stdout:1996 read 0x1eb4085 thru 0x1eb44a8 (0x424 bytes) 2024-09-12T07:20:15.928 INFO:teuthology.orchestra.run.smithi046.stdout:1997 read 0x19e3f86 thru 0x19ec58b (0x8606 bytes) 2024-09-12T07:20:15.929 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:15.929 INFO:teuthology.orchestra.run.smithi046.stdout:2000 writesame 0x2583014 thru 0x258316c data_size 0x5(0x159 bytes) 2024-09-12T07:20:15.953 INFO:teuthology.orchestra.run.smithi046.stdout:2001 punch from 0x2e607d8 to 0x2e641c6, (0x39ee bytes) 2024-09-12T07:20:15.955 INFO:teuthology.orchestra.run.smithi046.stdout:2002 trunc from 0x329f529 to 0x1aadb89 2024-09-12T07:20:15.957 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:15.958+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:16.034 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:16.034 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:16.043 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:16.045 INFO:teuthology.orchestra.run.smithi046.stdout:2003 write 0xe09bfac thru 0xe0a26bb (0x6710 bytes) 2024-09-12T07:20:16.049 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:16.061 INFO:teuthology.orchestra.run.smithi046.stdout:2006 trunc from 0xe0a26bc to 0x3b93d02 2024-09-12T07:20:16.071 INFO:teuthology.orchestra.run.smithi046.stdout:2007 writesame 0x307d07d thru 0x308132a data_size 0xf(0x42ae bytes) 2024-09-12T07:20:16.075 INFO:teuthology.orchestra.run.smithi046.stdout:2008 read 0x216cf16 thru 0x2177e17 (0xaf02 bytes) 2024-09-12T07:20:16.075 INFO:teuthology.orchestra.run.smithi046.stdout:2009 read 0x64e679 thru 0x65d3c3 (0xed4b bytes) 2024-09-12T07:20:16.086 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:16.088+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:16.093 INFO:teuthology.orchestra.run.smithi046.stdout:2010 write 0xdf4df6c thru 0xdf4f066 (0x10fb bytes) 2024-09-12T07:20:16.096 INFO:teuthology.orchestra.run.smithi046.stdout:2011 read 0x97ca87a thru 0x97d4897 (0xa01e bytes) 2024-09-12T07:20:16.097 INFO:teuthology.orchestra.run.smithi046.stdout:2012 write 0xa734d6e thru 0xa73638f (0x1622 bytes) 2024-09-12T07:20:16.108 INFO:teuthology.orchestra.run.smithi046.stdout:2013 trunc from 0xdf4f067 to 0x5b7cfaf 2024-09-12T07:20:16.118 INFO:teuthology.orchestra.run.smithi046.stdout:2014 read 0x10dcf1d thru 0x10e4558 (0x763c bytes) 2024-09-12T07:20:16.119 INFO:teuthology.orchestra.run.smithi046.stdout:2015 write 0x79da53 thru 0x7a8bbf (0xb16d bytes) 2024-09-12T07:20:16.123 INFO:teuthology.orchestra.run.smithi046.stdout:2016 read 0x3e27e47 thru 0x3e2ce03 (0x4fbd bytes) 2024-09-12T07:20:16.123 INFO:teuthology.orchestra.run.smithi046.stdout:2017 punch from 0xc06bc4 to 0xc0b97f, (0x4dbb bytes) 2024-09-12T07:20:16.131 INFO:teuthology.orchestra.run.smithi046.stdout:2018 write 0x7ea446d thru 0x7ea4c96 (0x82a bytes) 2024-09-12T07:20:16.134 INFO:teuthology.orchestra.run.smithi046.stdout:2019 read 0x349e21b thru 0x34a30f6 (0x4edc bytes) 2024-09-12T07:20:16.134 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:16.134 INFO:teuthology.orchestra.run.smithi046.stdout:2021 write 0x5b06952 thru 0x5b10ada (0xa189 bytes) 2024-09-12T07:20:16.141 INFO:teuthology.orchestra.run.smithi046.stdout:2022 read 0x2d1f650 thru 0x2d2e24f (0xec00 bytes) 2024-09-12T07:20:16.150 INFO:teuthology.orchestra.run.smithi046.stdout:2023 trunc from 0x7ea4c97 to 0xeccb53a 2024-09-12T07:20:16.155 INFO:teuthology.orchestra.run.smithi046.stdout:2024 read 0x59f6512 thru 0x5a02fce (0xcabd bytes) 2024-09-12T07:20:16.156 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:16.156 INFO:teuthology.orchestra.run.smithi046.stdout:2028 read 0x2bb32c3 thru 0x2bb6f73 (0x3cb1 bytes) 2024-09-12T07:20:16.156 INFO:teuthology.orchestra.run.smithi046.stdout:2030 read 0x41c7914 thru 0x41d424b (0xc938 bytes) 2024-09-12T07:20:16.157 INFO:teuthology.orchestra.run.smithi046.stdout:2031 write 0x9c643b4 thru 0x9c6fb0b (0xb758 bytes) 2024-09-12T07:20:16.158 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:16.159+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:16.159 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:16.159 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:16.161 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:16.161 INFO:teuthology.orchestra.run.smithi046.stdout:2034 write 0x421905 thru 0x42f387 (0xda83 bytes) 2024-09-12T07:20:16.164 INFO:teuthology.orchestra.run.smithi046.stdout:2035 punch from 0x15977c0 to 0x15a03ee, (0x8c2e bytes) 2024-09-12T07:20:16.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:16.170 INFO:teuthology.orchestra.run.smithi046.stdout:2036 trunc from 0xeccb53a to 0x8ce5870 2024-09-12T07:20:16.181 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:16.181 INFO:teuthology.orchestra.run.smithi046.stdout:2038 punch from 0x135fc07 to 0x13657bd, (0x5bb6 bytes) 2024-09-12T07:20:16.181 INFO:teuthology.orchestra.run.smithi046.stdout:2040 writesame 0x1325cfe thru 0x13273f5 data_size 0x3c(0x16f8 bytes) 2024-09-12T07:20:16.182 INFO:teuthology.orchestra.run.smithi046.stdout:2041 read 0x5b26528 thru 0x5b33e8e (0xd967 bytes) 2024-09-12T07:20:16.189 INFO:teuthology.orchestra.run.smithi046.stdout:2042 write 0x9fb66ca thru 0x9fb9513 (0x2e4a bytes) 2024-09-12T07:20:16.195 INFO:teuthology.orchestra.run.smithi046.stdout:2043 writesame 0x1dc4ec5 thru 0x1dcecd1 data_size 0x1(0x9e0d bytes) 2024-09-12T07:20:16.204 INFO:teuthology.orchestra.run.smithi046.stdout:2044 read 0x8cb51d6 thru 0x8cc3048 (0xde73 bytes) 2024-09-12T07:20:16.204 INFO:teuthology.orchestra.run.smithi046.stdout:2046 read 0x6f425bf thru 0x6f51cbb (0xf6fd bytes) 2024-09-12T07:20:16.207 INFO:teuthology.orchestra.run.smithi046.stdout:2048 trunc from 0x9fb9514 to 0xce159ac 2024-09-12T07:20:16.212 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:16.212 INFO:teuthology.orchestra.run.smithi046.stdout:2050 punch from 0x859fd0d to 0x85aea34, (0xed27 bytes) 2024-09-12T07:20:16.215 INFO:teuthology.orchestra.run.smithi046.stdout:2051 trunc from 0xce159ac to 0xa9695c3 2024-09-12T07:20:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.5 to 1 (10000) 2024-09-12T07:20:16.228 INFO:teuthology.orchestra.run.smithi046.stdout:2052 punch from 0x8418f02 to 0x8420347, (0x7445 bytes) 2024-09-12T07:20:16.229 INFO:teuthology.orchestra.run.smithi046.stdout:2053 write 0x37967d8 thru 0x37a57d0 (0xeff9 bytes) 2024-09-12T07:20:16.235 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-12T07:20:16.243 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:16.243 INFO:teuthology.orchestra.run.smithi046.stdout:2055 punch from 0x14ec229 to 0x14f72d9, (0xb0b0 bytes) 2024-09-12T07:20:16.243 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:16.243 INFO:teuthology.orchestra.run.smithi046.stdout:2057 trunc from 0xa9695c3 to 0x9d66827 2024-09-12T07:20:16.250 INFO:teuthology.orchestra.run.smithi046.stdout:2058 writesame 0x21f2825 thru 0x21f9974 data_size 0x25(0x7150 bytes) 2024-09-12T07:20:16.253 INFO:teuthology.orchestra.run.smithi046.stdout:2059 read 0x482a47d thru 0x4833b8e (0x9712 bytes) 2024-09-12T07:20:16.253 INFO:teuthology.orchestra.run.smithi046.stdout:2060 writesame 0x44b01a8 thru 0x44b7393 data_size 0x2e(0x71ec bytes) 2024-09-12T07:20:16.257 INFO:teuthology.orchestra.run.smithi046.stdout:2061 writesame 0x912bbd4 thru 0x912d51f data_size 0x2(0x194c bytes) 2024-09-12T07:20:16.259 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:16.260+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:16.260 INFO:teuthology.orchestra.run.smithi046.stdout:2062 writesame 0x24087dd thru 0x240afdf data_size 0x1(0x2803 bytes) 2024-09-12T07:20:16.265 INFO:teuthology.orchestra.run.smithi046.stdout:2063 read 0x678d037 thru 0x67932c8 (0x6292 bytes) 2024-09-12T07:20:16.265 INFO:teuthology.orchestra.run.smithi046.stdout:2065 punch from 0x6a3165f to 0x6a3ade0, (0x9781 bytes) 2024-09-12T07:20:16.265 INFO:teuthology.orchestra.run.smithi046.stdout:2066 read 0x494268a thru 0x494e108 (0xba7f bytes) 2024-09-12T07:20:16.265 INFO:teuthology.orchestra.run.smithi046.stdout:2067 write 0x4cbe85b thru 0x4cc4300 (0x5aa6 bytes) 2024-09-12T07:20:16.269 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:16.278 INFO:teuthology.orchestra.run.smithi046.stdout:2070 writesame 0xd816879 thru 0xd81f86f data_size 0x51(0x8ff7 bytes) 2024-09-12T07:20:16.290 INFO:teuthology.orchestra.run.smithi046.stdout:2071 trunc from 0xd81f870 to 0x36e7b1e 2024-09-12T07:20:16.302 INFO:teuthology.orchestra.run.smithi046.stdout:2072 read 0x118a5b7 thru 0x1196a61 (0xc4ab bytes) 2024-09-12T07:20:16.305 INFO:teuthology.orchestra.run.smithi046.stdout:2073 trunc from 0x36e7b1e to 0x4cd9c75 2024-09-12T07:20:16.310 INFO:teuthology.orchestra.run.smithi046.stdout:2074 read 0x1b8fcb1 thru 0x1b93eae (0x41fe bytes) 2024-09-12T07:20:16.314 INFO:teuthology.orchestra.run.smithi046.stdout:2075 trunc from 0x4cd9c75 to 0x7c01224 2024-09-12T07:20:16.319 INFO:teuthology.orchestra.run.smithi046.stdout:2076 read 0x22b1a2 thru 0x235950 (0xa7af bytes) 2024-09-12T07:20:16.320 INFO:teuthology.orchestra.run.smithi046.stdout:2077 writesame 0x5e77932 thru 0x5e84a5e data_size 0x1(0xd12d bytes) 2024-09-12T07:20:16.331 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:16.331 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:16.340 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:16.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:20:16.348 INFO:teuthology.orchestra.run.smithi046.stdout:2078 write 0x8965f9a thru 0x896690a (0x971 bytes) 2024-09-12T07:20:16.352 INFO:teuthology.orchestra.run.smithi046.stdout:2079 punch from 0x67d62b6 to 0x67d70b3, (0xdfd bytes) 2024-09-12T07:20:16.352 INFO:teuthology.orchestra.run.smithi046.stdout:2080 punch from 0x41d0151 to 0x41db5be, (0xb46d bytes) 2024-09-12T07:20:16.352 INFO:teuthology.orchestra.run.smithi046.stdout:2081 write 0x59849ae thru 0x598badf (0x7132 bytes) 2024-09-12T07:20:16.353 INFO:teuthology.orchestra.run.smithi046.stdout:2082 write 0x2578816 thru 0x2584656 (0xbe41 bytes) 2024-09-12T07:20:16.356 INFO:teuthology.orchestra.run.smithi046.stdout:2083 write 0x56475b1 thru 0x565614e (0xeb9e bytes) 2024-09-12T07:20:16.360 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:16.361+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:16.361 INFO:teuthology.orchestra.run.smithi046.stdout:2084 trunc from 0x896690b to 0xa5eeb5f 2024-09-12T07:20:16.367 INFO:teuthology.orchestra.run.smithi046.stdout:2085 read 0xa3335ea thru 0xa34021e (0xcc35 bytes) 2024-09-12T07:20:16.367 INFO:teuthology.orchestra.run.smithi046.stdout:2086 punch from 0x52656c to 0x533b1f, (0xd5b3 bytes) 2024-09-12T07:20:16.367 INFO:teuthology.orchestra.run.smithi046.stdout:2087 writesame 0x4c9a705 thru 0x4ca0431 data_size 0x3(0x5d2d bytes) 2024-09-12T07:20:16.372 INFO:teuthology.orchestra.run.smithi046.stdout:2088 read 0x4dfd187 thru 0x4dfea90 (0x190a bytes) 2024-09-12T07:20:16.373 INFO:teuthology.orchestra.run.smithi046.stdout:2090 write 0x9ecf388 thru 0x9ed7b44 (0x87bd bytes) 2024-09-12T07:20:16.381 INFO:teuthology.orchestra.run.smithi046.stdout:2093 write 0x7bdeb3 thru 0x7cd180 (0xf2ce bytes) 2024-09-12T07:20:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:2094 write 0x3a5b48f thru 0x3a65292 (0x9e04 bytes) 2024-09-12T07:20:16.392 INFO:teuthology.orchestra.run.smithi046.stdout:2095 write 0xa686f2f thru 0xa68ad96 (0x3e68 bytes) 2024-09-12T07:20:16.396 INFO:teuthology.orchestra.run.smithi046.stdout:2096 read 0x21f648b thru 0x21f8b0d (0x2683 bytes) 2024-09-12T07:20:16.397 INFO:teuthology.orchestra.run.smithi046.stdout:2098 punch from 0x55468a9 to 0x55502d5, (0x9a2c bytes) 2024-09-12T07:20:16.397 INFO:teuthology.orchestra.run.smithi046.stdout:2104 read 0x2fbc2a0 thru 0x2fc3685 (0x73e6 bytes) 2024-09-12T07:20:16.397 INFO:teuthology.orchestra.run.smithi046.stdout:2105 read 0x7223abe thru 0x722f0b4 (0xb5f7 bytes) 2024-09-12T07:20:16.404 INFO:teuthology.orchestra.run.smithi046.stdout:2106 write 0xb9660ce thru 0xb96bb9c (0x5acf bytes) 2024-09-12T07:20:16.407 INFO:teuthology.orchestra.run.smithi046.stdout:2107 read 0x1a468e3 thru 0x1a48edb (0x25f9 bytes) 2024-09-12T07:20:16.408 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:16.408 INFO:teuthology.orchestra.run.smithi046.stdout:2109 clone 16 order 23 su 262144 sc 2 2024-09-12T07:20:16.455 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:16.456 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:16.461 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:16.462+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:16.464 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:16.561 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:16.563+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:16.571 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:16.571 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:16.579 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:16.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:16.662 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:16.664+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:16.704 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:16.721 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:16.722 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:16.726 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:16.726 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:16.735 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:16.738 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:16.753 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:16.753 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:16.770 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:16.770 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:16.778 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:16.780+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:16.787 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:16.803 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:16.822 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:16.822 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:16.843 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:16.843 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:16.843 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:16.843 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:16.843 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:16.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:16.893 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:16.894+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:16.965 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:16.967+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:16.999 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:17.000 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:17.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:17.000 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:17.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.000 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.000 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:17.000 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451481591, 2024-09-12T07:20:17.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:20:17.000 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.000 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.000 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.001 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.001 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.001 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.001 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:20:17.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.001 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:17.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:20:17.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:20:17.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:20:17.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:20:17.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:20:17.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:20:17.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:20:17.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:20:17.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:20:17.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:20:17.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:20:17.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:20:17.005 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.005 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.005 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.005 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.005 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:17.005 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451461053999999, 2024-09-12T07:20:17.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:20:17.005 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:20:17.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451441051, 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:17.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:20:17.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:20:17.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:20:17.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:20:17.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.015 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.015 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:17.015 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451421101999999, 2024-09-12T07:20:17.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:20:17.015 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.015 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.015 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.015 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.015 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.019 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:17.020 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451400233999999, 2024-09-12T07:20:17.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:20:17.020 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.020 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.020 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.020 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.020 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.020 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.020 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:20:17.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:17.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:20:17.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:20:17.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:20:17.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:20:17.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:20:17.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:20:17.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:17.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:20:17.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:20:17.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.025 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451378533, 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:20:17.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:20:17.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:20:17.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:20:17.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:20:17.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:20:17.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451358093, 2024-09-12T07:20:17.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:17.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451330371999999, 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.035 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451307437000001, 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.039 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:17.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:20:17.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:20:17.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:20:17.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:20:17.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:20:17.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:20:17.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:20:17.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:20:17.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:20:17.044 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.044 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.045 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.045 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:17.045 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451287137, 2024-09-12T07:20:17.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:20:17.046 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.046 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.046 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.046 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.046 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.046 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.047 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:20:17.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.047 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:17.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:17.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:17.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:17.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:20:17.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:20:17.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:20:17.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:20:17.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:20:17.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:20:17.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:20:17.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:20:17.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:20:17.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:20:17.055 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.055 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.055 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.056 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.056 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:17.056 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451266861000001, 2024-09-12T07:20:17.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:20:17.057 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.057 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.057 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.058 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.058 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:20:17.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.058 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:17.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:17.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:17.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:20:17.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:20:17.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:20:17.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:20:17.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:20:17.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:20:17.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:20:17.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:20:17.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:20:17.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:20:17.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:20:17.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:20:17.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:20:17.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.066 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.067 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.067 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:17.067 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451246702000001, 2024-09-12T07:20:17.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:20:17.068 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.068 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.068 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.068 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.068 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.069 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.069 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:20:17.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.069 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:17.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:17.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:20:17.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:20:17.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:20:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:20:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:20:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:20:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:20:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:20:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:20:17.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:20:17.074 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.074 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.074 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.074 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.074 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:17.074 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451109504, 2024-09-12T07:20:17.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:20:17.074 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:20:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.077 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:20:17.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:20:17.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:20:17.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:20:17.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:20:17.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:20:17.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.079 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.451086856, 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:20:17.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:20:17.083 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68088 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 393216~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.444644096999999, 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10346986399999999, 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.084 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68088 2024-09-12T07:20:17.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.085 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:17.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:17.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:17.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:20:17.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555923+0000", 2024-09-12T07:20:17.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.891e-06 2024-09-12T07:20:17.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555929+0000", 2024-09-12T07:20:17.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0549999999999996e-06 2024-09-12T07:20:17.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555934+0000", 2024-09-12T07:20:17.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6419999999999998e-06 2024-09-12T07:20:17.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659060+0000", 2024-09-12T07:20:17.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103125862 2024-09-12T07:20:17.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659139+0000", 2024-09-12T07:20:17.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9450000000000007e-05 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659391+0000", 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025158400000000001 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:17.089 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.44462487, 2024-09-12T07:20:17.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:20:17.090 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.090 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.090 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.090 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.090 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.090 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.090 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:20:17.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.090 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:17.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:17.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:20:17.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.444395233, 2024-09-12T07:20:17.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:17.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:20:17.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:20:17.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.437471542000001, 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:17.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:20:17.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:20:17.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:20:17.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:20:17.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:20:17.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:20:17.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:20:17.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.437111958999999, 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.104 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:20:17.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:20:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:20:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:20:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:20:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:20:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.108 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.437031973, 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:17.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:17.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:20:17.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:17.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:20:17.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:20:17.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:17.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:17.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:20:17.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:20:17.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:17.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:17.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:17.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:20:17.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:20:17.114 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.114 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.114 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.114 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:17.114 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:17.114 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:17.116 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:17.118+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:17.116 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:17.217 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:17.218+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:17.226 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:17.227 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:17.227 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.6 to 1 (10000) 2024-09-12T07:20:17.228 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:20:17.242 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:17.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-12T07:20:17.295 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone15 from 0xb96bb9d (overlap 0x1aadb89) to 0x555f1b 2024-09-12T07:20:17.318 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:17.319+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:17.369 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:17.369 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:17.369 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:17.369 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:17.369 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:17.377 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-12T07:20:17.382 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:17.419 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:17.419+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:17.519 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:17.520+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:17.540 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82749 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 429056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8971428889999995, 2024-09-12T07:20:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055052166, 2024-09-12T07:20:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82749 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0800000000000001e-07 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644459+0000", 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1769999999999999e-06 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644462+0000", 2024-09-12T07:20:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8779999999999998e-06 2024-09-12T07:20:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644465+0000", 2024-09-12T07:20:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6509999999999999e-06 2024-09-12T07:20:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699279+0000", 2024-09-12T07:20:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054813832999999999 2024-09-12T07:20:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699359+0000", 2024-09-12T07:20:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0150999999999999e-05 2024-09-12T07:20:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699510+0000", 2024-09-12T07:20:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015026800000000001 2024-09-12T07:20:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82750 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 432128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8971300049999993, 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055234722, 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82750 2024-09-12T07:20:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644471+0000", 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.17e-07 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644472+0000", 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.19e-06 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644482+0000", 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0332e-05 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644489+0000", 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7039999999999997e-06 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699549+0000", 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055060588000000001 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699592+0000", 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2707000000000003e-05 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699705+0000", 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112984 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82751 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 436224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8971026510000009, 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055415691000000003, 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82751 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644846+0000", 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034807600000000001 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644850+0000", 2024-09-12T07:20:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1649999999999999e-06 2024-09-12T07:20:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644860+0000", 2024-09-12T07:20:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6949999999999995e-06 2024-09-12T07:20:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644866+0000", 2024-09-12T07:20:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3640000000000004e-06 2024-09-12T07:20:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699735+0000", 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054868888999999997 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699776+0000", 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0722000000000001e-05 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699913+0000", 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013778 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82752 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 439296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8967249079999995, 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226917, 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82752 2024-09-12T07:20:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644878+0000", 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7850000000000001e-06 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644882+0000", 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9990000000000002e-06 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644885+0000", 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.965e-06 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699952+0000", 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067243000000002 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699989+0000", 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7450999999999998e-05 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700102+0000", 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113124 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82753 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 443392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8967122520000004, 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055422102000000001, 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82753 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645016+0000", 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000127847 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645024+0000", 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7419999999999997e-06 2024-09-12T07:20:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645030+0000", 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7229999999999998e-06 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645034+0000", 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9380000000000002e-06 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700132+0000", 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055097699 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700171+0000", 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9093e-05 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700310+0000", 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013905999999999999 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82754 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 446464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8965563450000005, 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055424703999999998, 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82754 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645318+0000", 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027397499999999998 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645322+0000", 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.766e-06 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645331+0000", 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2140000000000002e-06 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645338+0000", 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0929999999999997e-06 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700330+0000", 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054991384999999997 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700369+0000", 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700469+0000", 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9913000000000004e-05 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82755 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 449536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8962513550000004, 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055388918000000002, 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82755 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645500+0000", 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015050999999999999 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645502+0000", 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645509+0000", 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3969999999999999e-06 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645513+0000", 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9230000000000004e-06 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700507+0000", 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054994692999999997 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700583+0000", 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5656999999999997e-05 2024-09-12T07:20:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700738+0000", 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154869 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82756 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 453632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8960804670000009, 2024-09-12T07:20:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055451746000000003, 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82756 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645763+0000", 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000243105 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645766+0000", 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0750000000000002e-06 2024-09-12T07:20:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645774+0000", 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9370000000000004e-06 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645783+0000", 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5159999999999994e-06 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700780+0000", 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054997329999999997 2024-09-12T07:20:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700835+0000", 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4789e-05 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700972+0000", 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013699400000000001 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82757 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8958060989999996, 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055382590000000002, 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82757 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645926+0000", 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131979 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645929+0000", 2024-09-12T07:20:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0199999999999999e-06 2024-09-12T07:20:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645935+0000", 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645939+0000", 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.924e-06 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701007+0000", 2024-09-12T07:20:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067374000000002 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701043+0000", 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6325999999999997e-05 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701177+0000", 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013387300000000001 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82758 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 460800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8956487059999994, 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055417890999999997, 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82758 2024-09-12T07:20:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646164+0000", 2024-09-12T07:20:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000212189 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646167+0000", 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646173+0000", 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4139999999999999e-06 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646177+0000", 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701208+0000", 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055031322000000001 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701254+0000", 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5701999999999997e-05 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701370+0000", 2024-09-12T07:20:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115456 2024-09-12T07:20:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82759 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 463872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8954161450000004, 2024-09-12T07:20:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055376838999999997, 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82759 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646368+0000", 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183399 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646372+0000", 2024-09-12T07:20:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9190000000000001e-06 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646380+0000", 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7819999999999999e-06 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646384+0000", 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701407+0000", 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055022763000000002 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701455+0000", 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7660000000000001e-05 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701561+0000", 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010642899999999999 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82760 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8952085959999998, 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055355572999999998, 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82760 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646631+0000", 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000238798 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646634+0000", 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.016e-06 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646640+0000", 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2750000000000001e-06 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646644+0000", 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7859999999999998e-06 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701599+0000", 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054955693999999999 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701638+0000", 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8319999999999999e-05 2024-09-12T07:20:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701747+0000", 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109684 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82761 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8949499670000005, 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055296990999999997, 2024-09-12T07:20:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82761 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646756+0000", 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105899 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646759+0000", 2024-09-12T07:20:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8380000000000002e-06 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646765+0000", 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646769+0000", 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7900000000000001e-06 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701785+0000", 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055015888999999998 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701832+0000", 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6514000000000002e-05 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701947+0000", 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115833 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82762 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 474112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8948243999999992, 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055346863000000003, 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82762 2024-09-12T07:20:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647063+0000", 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000287115 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647066+0000", 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0189999999999998e-06 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647075+0000", 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7099999999999996e-06 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647081+0000", 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6790000000000002e-06 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701977+0000", 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054896665999999997 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702015+0000", 2024-09-12T07:20:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.79e-05 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702123+0000", 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107774 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82763 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8945094830000002, 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226509999999999, 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82763 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647236+0000", 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000144597 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647239+0000", 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1329999999999999e-06 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647245+0000", 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647249+0000", 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2270000000000004e-06 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702160+0000", 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054910549000000003 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702205+0000", 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5575000000000003e-05 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702317+0000", 2024-09-12T07:20:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011194899999999999 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82764 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 481280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8943437319999994, 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055229250000000001, 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82764 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647471+0000", 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000214625 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647474+0000", 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8509999999999999e-06 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647480+0000", 2024-09-12T07:20:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.28e-06 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647484+0000", 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.771e-06 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702346+0000", 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054861551000000001 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702382+0000", 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.65e-05 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702486+0000", 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010367200000000001 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82765 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 485376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8941081850000003, 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055184729000000002, 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82765 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647681+0000", 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188965 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647693+0000", 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.225e-05 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647700+0000", 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9929999999999998e-06 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647704+0000", 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9469999999999996e-06 2024-09-12T07:20:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702522+0000", 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054817346000000003 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702566+0000", 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4672999999999997e-05 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702677+0000", 2024-09-12T07:20:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110555 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82766 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 488448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8938889470000007, 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055132717999999997, 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82766 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647959+0000", 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024739599999999999 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647964+0000", 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9409999999999998e-06 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647975+0000", 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1404e-05 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647983+0000", 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2969999999999998e-06 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702706+0000", 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054723690999999998 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702743+0000", 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6464999999999997e-05 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702844+0000", 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101524 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82767 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 492544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8936049060000002, 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055054448999999998, 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82767 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648182+0000", 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018652299999999999 2024-09-12T07:20:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648185+0000", 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3289999999999998e-06 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648192+0000", 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7159999999999997e-06 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648196+0000", 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1439999999999996e-06 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702873+0000", 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054676356000000002 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702930+0000", 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7643999999999999e-05 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703050+0000", 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119737 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82768 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 495616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.648206+0000", 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.8933945229999996, 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055030120000000002, 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82768 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648206+0000", 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648206+0000", 2024-09-12T07:20:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648400+0000", 2024-09-12T07:20:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019423500000000001 2024-09-12T07:20:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648404+0000", 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1019999999999999e-06 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648412+0000", 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0669999999999992e-06 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648419+0000", 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2539999999999998e-06 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703088+0000", 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054669199000000002 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703125+0000", 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6847999999999997e-05 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703236+0000", 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000111415 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:17.632 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:17.632 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:17.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:17.634 INFO:teuthology.orchestra.run.smithi046.stdout:2110 read 0x6656e29 thru 0x6660f53 (0xa12b bytes) 2024-09-12T07:20:17.634 INFO:teuthology.orchestra.run.smithi046.stdout:2111 read 0x49c8ec4 thru 0x49d3dc8 (0xaf05 bytes) 2024-09-12T07:20:17.634 INFO:teuthology.orchestra.run.smithi046.stdout:2112 writesame 0x3b505d2 thru 0x3b52bca data_size 0x1(0x25f9 bytes) 2024-09-12T07:20:17.634 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:17.634+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:17.637 INFO:teuthology.orchestra.run.smithi046.stdout:2113 punch from 0x1b279fc to 0x1b35420, (0xda24 bytes) 2024-09-12T07:20:17.637 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:17.637 INFO:teuthology.orchestra.run.smithi046.stdout:2116 read 0x6b23b1a thru 0x6b254ba (0x19a1 bytes) 2024-09-12T07:20:17.637 INFO:teuthology.orchestra.run.smithi046.stdout:2118 read 0x8f164d8 thru 0x8f21f13 (0xba3c bytes) 2024-09-12T07:20:17.637 INFO:teuthology.orchestra.run.smithi046.stdout:2119 write 0x5570270 thru 0x55728d5 (0x2666 bytes) 2024-09-12T07:20:17.681 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:17.681 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:17.681 INFO:teuthology.orchestra.run.smithi046.stdout:2123 punch from 0x8ea7b9e to 0x8eb6e43, (0xf2a5 bytes) 2024-09-12T07:20:17.681 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:17.684 INFO:teuthology.orchestra.run.smithi046.stdout:2125 trunc from 0xb96bb9d to 0x8d99ccf 2024-09-12T07:20:17.704 INFO:teuthology.orchestra.run.smithi046.stdout:2126 writesame 0x52e99d9 thru 0x52f752a data_size 0x43(0xdb52 bytes) 2024-09-12T07:20:17.706 INFO:teuthology.orchestra.run.smithi046.stdout:2128 writesame 0x5163fd4 thru 0x5166541 data_size 0x3(0x256e bytes) 2024-09-12T07:20:17.709 INFO:teuthology.orchestra.run.smithi046.stdout:2129 write 0x56067e4 thru 0x560c71f (0x5f3c bytes) 2024-09-12T07:20:17.734 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:17.734+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:17.750 INFO:teuthology.orchestra.run.smithi046.stdout:2130 read 0x69e4ee1 thru 0x69ec8e4 (0x7a04 bytes) 2024-09-12T07:20:17.763 INFO:teuthology.orchestra.run.smithi046.stdout:2131 write 0xe9731f7 thru 0xe97c9d4 (0x97de bytes) 2024-09-12T07:20:17.766 INFO:teuthology.orchestra.run.smithi046.stdout:2132 read 0x1dc5378 thru 0x1dcd15e (0x7de7 bytes) 2024-09-12T07:20:17.767 INFO:teuthology.orchestra.run.smithi046.stdout:2133 write 0xae7029 thru 0xae85d6 (0x15ae bytes) 2024-09-12T07:20:17.779 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:17.779 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:17.779 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:17.779 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:17.788 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:17.834 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:17.835+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:17.844 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:17.856 INFO:teuthology.orchestra.run.smithi046.stdout:2136 trunc from 0xe97c9d5 to 0x4072400 2024-09-12T07:20:17.905 INFO:teuthology.orchestra.run.smithi046.stdout:2138 write 0xb5e44c0 thru 0xb5e5c72 (0x17b3 bytes) 2024-09-12T07:20:17.908 INFO:teuthology.orchestra.run.smithi046.stdout:2140 writesame 0x45fac9c thru 0x46016b2 data_size 0xb(0x6a17 bytes) 2024-09-12T07:20:17.910 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:17.910 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:17.910 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:17.910 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:17.910 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:17.916 INFO:teuthology.orchestra.run.smithi046.stdout:2141 write 0xc375023 thru 0xc3791c9 (0x41a7 bytes) 2024-09-12T07:20:17.919 INFO:teuthology.orchestra.run.smithi046.stdout:2142 read 0xa0ee65d thru 0xa0f5dab (0x774f bytes) 2024-09-12T07:20:17.919 INFO:teuthology.orchestra.run.smithi046.stdout:2143 read 0x3b56644 thru 0x3b58a8d (0x244a bytes) 2024-09-12T07:20:17.920 INFO:teuthology.orchestra.run.smithi046.stdout:2144 read 0x6d193c5 thru 0x6d25448 (0xc084 bytes) 2024-09-12T07:20:17.920 INFO:teuthology.orchestra.run.smithi046.stdout:2146 write 0x8c62762 thru 0x8c63cab (0x154a bytes) 2024-09-12T07:20:17.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:17.930 INFO:teuthology.orchestra.run.smithi046.stdout:2147 trunc from 0xc3791ca to 0x5697104 2024-09-12T07:20:17.935 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:17.936+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:17.943 INFO:teuthology.orchestra.run.smithi046.stdout:2151 trunc from 0x5697104 to 0x8716b8b 2024-09-12T07:20:17.962 INFO:teuthology.orchestra.run.smithi046.stdout:2152 write 0xeb32c09 thru 0xeb406a2 (0xda9a bytes) 2024-09-12T07:20:17.965 INFO:teuthology.orchestra.run.smithi046.stdout:2153 read 0x3d8e8c7 thru 0x3d940ae (0x57e8 bytes) 2024-09-12T07:20:17.981 INFO:teuthology.orchestra.run.smithi046.stdout:2154 trunc from 0xeb406a3 to 0x5b3a5b 2024-09-12T07:20:18.036 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:18.038+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:18.044 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:18.044 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:18.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:18.044 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:18.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.045 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:996 3.0 3:05cac305:::rbd_data.116011b0bca6.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:20:18.045 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.308560+0000", 2024-09-12T07:20:18.045 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.736501531999998, 2024-09-12T07:20:18.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014904776, 2024-09-12T07:20:18.045 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:18.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 996 2024-09-12T07:20:18.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.046 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308560+0000", 2024-09-12T07:20:18.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308560+0000", 2024-09-12T07:20:18.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308561+0000", 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.536e-06 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308570+0000", 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1090000000000004e-06 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308584+0000", 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3893000000000001e-05 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308592+0000", 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1009999999999992e-06 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308644+0000", 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2117999999999998e-05 2024-09-12T07:20:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308754+0000", 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109666 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.310773+0000", 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020187310000000002 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.323365+0000", 2024-09-12T07:20:18.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012591836 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.323377+0000", 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2109999999999999e-05 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.323413+0000", 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6245999999999998e-05 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.323464+0000", 2024-09-12T07:20:18.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1430000000000001e-05 2024-09-12T07:20:18.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.051 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.051 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:18.051 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:18.051 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.661673287999999, 2024-09-12T07:20:18.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:20:18.051 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:18.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.052 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:20:18.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.052 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:20:18.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:20:18.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66600 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:18.057 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.389749+0000", 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.655312545999999, 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032114065999999997, 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66600 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389749+0000", 2024-09-12T07:20:18.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389749+0000", 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389750+0000", 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.421e-06 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389755+0000", 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8250000000000004e-06 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389770+0000", 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5659e-05 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389779+0000", 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4440000000000008e-06 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.401597+0000", 2024-09-12T07:20:18.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011818265 2024-09-12T07:20:18.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.401696+0000", 2024-09-12T07:20:18.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9097000000000003e-05 2024-09-12T07:20:18.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.402026+0000", 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032977499999999998 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.413052+0000", 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011026094 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421766+0000", 2024-09-12T07:20:18.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087136069999999999 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421822+0000", 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6332000000000002e-05 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421863+0000", 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0547000000000003e-05 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.064 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66677 3.0 3:07537075:::rbd_data.1160f022dd61.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:18.064 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.389797+0000", 2024-09-12T07:20:18.064 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.655264029, 2024-09-12T07:20:18.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022850626999999998, 2024-09-12T07:20:18.064 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.064 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:18.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66677 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389797+0000", 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389797+0000", 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389798+0000", 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0999999999999997e-07 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389802+0000", 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.817e-06 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389809+0000", 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4900000000000003e-06 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389815+0000", 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1099999999999999e-06 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.401560+0000", 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011744409000000001 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.401777+0000", 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000217615 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.402080+0000", 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030243799999999998 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.404470+0000", 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0023900010000000001 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412521+0000", 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080516579999999997 2024-09-12T07:20:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:18.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412587+0000", 2024-09-12T07:20:18.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5522e-05 2024-09-12T07:20:18.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412648+0000", 2024-09-12T07:20:18.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0757000000000001e-05 2024-09-12T07:20:18.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.069 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.598595024, 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:20:18.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:20:18.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:20:18.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:20:18.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:20:18.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.594889939, 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:18.076 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:18.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:20:18.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:20:18.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:20:18.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:20:18.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:20:18.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:20:18.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:20:18.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:20:18.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:18.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:20:18.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:20:18.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:18.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:20:18.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:20:18.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:18.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:20:18.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:20:18.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:18.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:20:18.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:20:18.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:20:18.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:20:18.083 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.083 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.083 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.083 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:69559 3.1a 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:18.083 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.681624+0000", 2024-09-12T07:20:18.083 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.363437017000001, 2024-09-12T07:20:18.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023561162, 2024-09-12T07:20:18.083 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 69559 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.681624+0000", 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.681624+0000", 2024-09-12T07:20:18.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.681626+0000", 2024-09-12T07:20:18.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5409999999999999e-06 2024-09-12T07:20:18.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683088+0000", 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001462009 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683122+0000", 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683133+0000", 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1559000000000001e-05 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683225+0000", 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2330000000000005e-05 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683325+0000", 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9513999999999996e-05 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683782+0000", 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00045661299999999997 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.691512+0000", 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0077308530000000002 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.705112+0000", 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013599862000000001 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.705155+0000", 2024-09-12T07:20:18.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2988999999999998e-05 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.705185+0000", 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.995e-05 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:69560 3.1a 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.683195+0000", 2024-09-12T07:20:18.089 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.361865778, 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024920327999999999, 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 69560 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683195+0000", 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683195+0000", 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683197+0000", 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.624e-06 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683202+0000", 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1189999999999996e-06 2024-09-12T07:20:18.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683892+0000", 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00068997999999999998 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683905+0000", 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3066e-05 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.687754+0000", 2024-09-12T07:20:18.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038485139999999999 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.687860+0000", 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106712 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.688125+0000", 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026430699999999998 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:18.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.692505+0000", 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0043800030000000004 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.708032+0000", 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01552693 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.708079+0000", 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7775999999999998e-05 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.708116+0000", 2024-09-12T07:20:18.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6297000000000002e-05 2024-09-12T07:20:18.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.096 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.096 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:69639 3.1a 3:5ec4a676:::rbd_data.1160f022dd61.0000000000000019:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:18.096 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.685937+0000", 2024-09-12T07:20:18.096 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.359124490999999, 2024-09-12T07:20:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022266522, 2024-09-12T07:20:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 69639 2024-09-12T07:20:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685937+0000", 2024-09-12T07:20:18.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685937+0000", 2024-09-12T07:20:18.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685938+0000", 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02e-06 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685951+0000", 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3621000000000001e-05 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685961+0000", 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0210999999999999e-05 2024-09-12T07:20:18.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685968+0000", 2024-09-12T07:20:18.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9040000000000003e-06 2024-09-12T07:20:18.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.691551+0000", 2024-09-12T07:20:18.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055829089999999996 2024-09-12T07:20:18.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.691668+0000", 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117186 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.708203+0000", 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016534671000000001 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78042 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:b [read 8323072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.101 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528554+0000", 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.516506720000001, 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011297491, 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78042 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528554+0000", 2024-09-12T07:20:18.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528554+0000", 2024-09-12T07:20:18.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528555+0000", 2024-09-12T07:20:18.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4200000000000001e-07 2024-09-12T07:20:18.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528557+0000", 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0480000000000001e-06 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528565+0000", 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2490000000000001e-06 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528578+0000", 2024-09-12T07:20:18.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2513e-05 2024-09-12T07:20:18.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539560+0000", 2024-09-12T07:20:18.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010982228 2024-09-12T07:20:18.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539642+0000", 2024-09-12T07:20:18.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1780000000000006e-05 2024-09-12T07:20:18.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539852+0000", 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021023099999999999 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78043 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 7798784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528586+0000", 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.516475475, 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011197245999999999, 2024-09-12T07:20:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78043 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528586+0000", 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528586+0000", 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528586+0000", 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9200000000000001e-07 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528588+0000", 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2709999999999999e-06 2024-09-12T07:20:18.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528597+0000", 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4130000000000006e-06 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528602+0000", 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2950000000000001e-06 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539546+0000", 2024-09-12T07:20:18.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010943575000000001 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539626+0000", 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0531999999999994e-05 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539783+0000", 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000156668 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78044 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 7929856~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528609+0000", 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.516451613999999, 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011471324, 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78044 2024-09-12T07:20:18.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.112 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528609+0000", 2024-09-12T07:20:18.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528609+0000", 2024-09-12T07:20:18.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528610+0000", 2024-09-12T07:20:18.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2199999999999999e-07 2024-09-12T07:20:18.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528612+0000", 2024-09-12T07:20:18.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.888e-06 2024-09-12T07:20:18.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528620+0000", 2024-09-12T07:20:18.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3140000000000004e-06 2024-09-12T07:20:18.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528625+0000", 2024-09-12T07:20:18.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.87e-06 2024-09-12T07:20:18.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539809+0000", 2024-09-12T07:20:18.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011183659 2024-09-12T07:20:18.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539865+0000", 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6563e-05 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540081+0000", 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000215608 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.116 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78045 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 8060928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.116 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528631+0000", 2024-09-12T07:20:18.116 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.516429714999999, 2024-09-12T07:20:18.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011695537000000001, 2024-09-12T07:20:18.116 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78045 2024-09-12T07:20:18.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.117 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528631+0000", 2024-09-12T07:20:18.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528631+0000", 2024-09-12T07:20:18.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528632+0000", 2024-09-12T07:20:18.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.46e-07 2024-09-12T07:20:18.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528634+0000", 2024-09-12T07:20:18.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9700000000000002e-06 2024-09-12T07:20:18.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528640+0000", 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7329999999999997e-06 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528646+0000", 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1699999999999996e-06 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540111+0000", 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011465444999999999 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540164+0000", 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2927999999999999e-05 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540327+0000", 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016294500000000001 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78046 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 8192000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528655+0000", 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.516406084, 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012006785000000001, 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78046 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.121 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528655+0000", 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528655+0000", 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528655+0000", 2024-09-12T07:20:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9999999999999998e-07 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528657+0000", 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.125e-06 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528664+0000", 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5660000000000003e-06 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528668+0000", 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2570000000000001e-06 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540386+0000", 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011717627 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540470+0000", 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3891e-05 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540662+0000", 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000191919 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78047 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 8323072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528677+0000", 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.516384056, 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012383072, 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78047 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528677+0000", 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528677+0000", 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528677+0000", 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.65e-07 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528680+0000", 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4449999999999999e-06 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528687+0000", 2024-09-12T07:20:18.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5329999999999997e-06 2024-09-12T07:20:18.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528692+0000", 2024-09-12T07:20:18.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-12T07:20:18.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540693+0000", 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012000613 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540781+0000", 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8178999999999997e-05 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.541060+0000", 2024-09-12T07:20:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027922400000000001 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78287 3.19 3:99bc29a5:::rbd_data.116087d6d6a2.0000000000000047:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.635267+0000", 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.409793677, 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022254682000000001, 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:18.130 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78287 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635267+0000", 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635267+0000", 2024-09-12T07:20:18.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635269+0000", 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.178e-06 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635273+0000", 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.121e-06 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635281+0000", 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5779999999999999e-06 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635286+0000", 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8199999999999996e-06 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635352+0000", 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5571e-05 2024-09-12T07:20:18.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635473+0000", 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000121795 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.639340+0000", 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038665679999999999 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.657349+0000", 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018008941000000001 2024-09-12T07:20:18.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.657402+0000", 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3142e-05 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.657466+0000", 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4325000000000005e-05 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.657522+0000", 2024-09-12T07:20:18.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5643000000000002e-05 2024-09-12T07:20:18.136 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.136 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.136 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.136 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78288 3.1a 3:58d7858f:::rbd_data.116087d6d6a2.0000000000000048:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.136 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.635707+0000", 2024-09-12T07:20:18.136 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.409353787000001, 2024-09-12T07:20:18.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016418782, 2024-09-12T07:20:18.136 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78288 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635707+0000", 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635707+0000", 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635708+0000", 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8100000000000002e-07 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635712+0000", 2024-09-12T07:20:18.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-12T07:20:18.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635721+0000", 2024-09-12T07:20:18.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6539999999999994e-06 2024-09-12T07:20:18.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635726+0000", 2024-09-12T07:20:18.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5580000000000002e-06 2024-09-12T07:20:18.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635768+0000", 2024-09-12T07:20:18.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1897999999999998e-05 2024-09-12T07:20:18.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635844+0000", 2024-09-12T07:20:18.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6446999999999995e-05 2024-09-12T07:20:18.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:18.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.638143+0000", 2024-09-12T07:20:18.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022985079999999999 2024-09-12T07:20:18.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:18.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.651990+0000", 2024-09-12T07:20:18.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013846712000000001 2024-09-12T07:20:18.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:18.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.652009+0000", 2024-09-12T07:20:18.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9548e-05 2024-09-12T07:20:18.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:18.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.652070+0000", 2024-09-12T07:20:18.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0523999999999999e-05 2024-09-12T07:20:18.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.652126+0000", 2024-09-12T07:20:18.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6462000000000003e-05 2024-09-12T07:20:18.142 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.143 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.143 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.143 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:20:18.143 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:18.143 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.59654366, 2024-09-12T07:20:18.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:20:18.143 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:20:18.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:20:18.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:88881 3.1a 3:5dc3e11f:::rbd_data.11607a56846.0000000000000016:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 2129920~15013] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:13.612488+0000", 2024-09-12T07:20:18.148 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.4325727009999998, 2024-09-12T07:20:18.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0163359, 2024-09-12T07:20:18.149 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.149 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:18.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 88881 2024-09-12T07:20:18.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.149 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.612488+0000", 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.612488+0000", 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.612492+0000", 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7139999999999999e-06 2024-09-12T07:20:18.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616402+0000", 2024-09-12T07:20:18.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039097059999999998 2024-09-12T07:20:18.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616427+0000", 2024-09-12T07:20:18.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5667999999999999e-05 2024-09-12T07:20:18.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616440+0000", 2024-09-12T07:20:18.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2495e-05 2024-09-12T07:20:18.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616525+0000", 2024-09-12T07:20:18.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5442e-05 2024-09-12T07:20:18.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616693+0000", 2024-09-12T07:20:18.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000167691 2024-09-12T07:20:18.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:18.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.617037+0000", 2024-09-12T07:20:18.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034386099999999999 2024-09-12T07:20:18.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:18.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.627285+0000", 2024-09-12T07:20:18.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010248554999999999 2024-09-12T07:20:18.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.628694+0000", 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001408379 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.628766+0000", 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1766000000000004e-05 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.628824+0000", 2024-09-12T07:20:18.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8622999999999998e-05 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:97141 3.6 e57/24 3:67e19a26:::rbd_data.1160b5642b3d.000000000000000b:head v 57'262, mlcod=57'260)", 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.666167+0000", 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.37889397600000002, 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016803757999999999, 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.666167+0000", 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.666167+0000", 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.666169+0000", 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.564e-06 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.669447+0000", 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0032787990000000002 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.669468+0000", 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0285e-05 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.669477+0000", 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3570000000000004e-06 2024-09-12T07:20:18.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.669553+0000", 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6026999999999996e-05 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.669578+0000", 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5320000000000002e-05 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.682897+0000", 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013318919 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.682971+0000", 2024-09-12T07:20:18.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3486999999999996e-05 2024-09-12T07:20:18.159 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.159 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.159 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.159 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.159 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.159 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:18.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:18.163 INFO:teuthology.orchestra.run.smithi046.stdout:2155 write 0x4a7db32 thru 0x4a80583 (0x2a52 bytes) 2024-09-12T07:20:18.163 INFO:teuthology.orchestra.run.smithi046.stdout:2156 read 0x18a1c5f thru 0x18a4c11 (0x2fb3 bytes) 2024-09-12T07:20:18.163 INFO:teuthology.orchestra.run.smithi046.stdout:2158 read 0x20cfa50 thru 0x20d3369 (0x391a bytes) 2024-09-12T07:20:18.163 INFO:teuthology.orchestra.run.smithi046.stdout:2159 read 0x446c5f4 thru 0x44788de (0xc2eb bytes) 2024-09-12T07:20:18.163 INFO:teuthology.orchestra.run.smithi046.stdout:2160 write 0x2e2e570 thru 0x2e3d985 (0xf416 bytes) 2024-09-12T07:20:18.163 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.163 INFO:teuthology.orchestra.run.smithi046.stdout:2162 read 0x50ccf2 thru 0x51aa45 (0xdd54 bytes) 2024-09-12T07:20:18.163 INFO:teuthology.orchestra.run.smithi046.stdout:2163 write 0xc855958 thru 0xc8589fd (0x30a6 bytes) 2024-09-12T07:20:18.163 INFO:teuthology.orchestra.run.smithi046.stdout:2164 write 0x2d5f8c1 thru 0x2d689d0 (0x9110 bytes) 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:2165 writesame 0x9374846 thru 0x9382c8c data_size 0x1(0xe447 bytes) 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:2166 trunc from 0xc8589fe to 0xa527ab3 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:2168 read 0x8369f8e thru 0x83782e2 (0xe355 bytes) 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:2172 trunc from 0xa527ab3 to 0x8e86ef9 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:2173 punch from 0x2be1253 to 0x2be74e9, (0x6296 bytes) 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:2176 write 0x6d14a8b thru 0x6d1a79c (0x5d12 bytes) 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:2178 read 0x212d594 thru 0x2132e29 (0x5896 bytes) 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:2179 write 0x62e0cca thru 0x62ea71e (0x9a55 bytes) 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:2182 punch from 0x3bfb658 to 0x3c03335, (0x7cdd bytes) 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:2183 read 0x51fa072 thru 0x5204b51 (0xaae0 bytes) 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:2184 punch from 0x441a4ef to 0x4423b0d, (0x961e bytes) 2024-09-12T07:20:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:2185 write 0x94e7b8c thru 0x94f5723 (0xdb98 bytes) 2024-09-12T07:20:18.165 INFO:teuthology.orchestra.run.smithi046.stdout:2186 read 0x3e892a0 thru 0x3e96405 (0xd166 bytes) 2024-09-12T07:20:18.165 INFO:teuthology.orchestra.run.smithi046.stdout:2187 write 0xd961b2b thru 0xd96471d (0x2bf3 bytes) 2024-09-12T07:20:18.165 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:18.161+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:18.169 INFO:teuthology.orchestra.run.smithi046.stdout:2190 trunc from 0xd96471e to 0x30ec513 2024-09-12T07:20:18.185 INFO:teuthology.orchestra.run.smithi046.stdout:2191 trunc from 0x30ec513 to 0x8711f6e 2024-09-12T07:20:18.190 INFO:teuthology.orchestra.run.smithi046.stdout:2192 punch from 0x5a8011f to 0x5a85f28, (0x5e09 bytes) 2024-09-12T07:20:18.202 INFO:teuthology.orchestra.run.smithi046.stdout:2194 writesame 0xe6256f9 thru 0xe627382 data_size 0xd(0x1c8a bytes) 2024-09-12T07:20:18.202 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.202 INFO:teuthology.orchestra.run.smithi046.stdout:2197 write 0xda18963 thru 0xda26c74 (0xe312 bytes) 2024-09-12T07:20:18.215 INFO:teuthology.orchestra.run.smithi046.stdout:2198 trunc from 0xe627383 to 0x55646d4 2024-09-12T07:20:18.223 INFO:teuthology.orchestra.run.smithi046.stdout:2199 write 0xa331c9 thru 0xa3bcc4 (0x8afc bytes) 2024-09-12T07:20:18.224 INFO:teuthology.orchestra.run.smithi046.stdout:2200 read 0x3a3849d thru 0x3a3be53 (0x39b7 bytes) 2024-09-12T07:20:18.225 INFO:teuthology.orchestra.run.smithi046.stdout:2201 trunc from 0x55646d4 to 0x47bc0ea 2024-09-12T07:20:18.231 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.231 INFO:teuthology.orchestra.run.smithi046.stdout:2203 punch from 0x419693 to 0x41d097, (0x3a04 bytes) 2024-09-12T07:20:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.7 to 1 (10000) 2024-09-12T07:20:18.253 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-12T07:20:18.254 INFO:teuthology.orchestra.run.smithi046.stdout:2206 write 0xea6058b thru 0xea6935e (0x8dd4 bytes) 2024-09-12T07:20:18.256 INFO:teuthology.orchestra.run.smithi046.stdout:2207 punch from 0xbfe0939 to 0xbfe35ff, (0x2cc6 bytes) 2024-09-12T07:20:18.266 INFO:teuthology.orchestra.run.smithi046.stdout:2208 trunc from 0xea6935f to 0x58907b0 2024-09-12T07:20:18.267 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:18.268+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:18.275 INFO:teuthology.orchestra.run.smithi046.stdout:2209 read 0x27b1b6a thru 0x27bc4f5 (0xa98c bytes) 2024-09-12T07:20:18.287 INFO:teuthology.orchestra.run.smithi046.stdout:2211 write 0xa976a60 thru 0xa9791c4 (0x2765 bytes) 2024-09-12T07:20:18.291 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.292 INFO:teuthology.orchestra.run.smithi046.stdout:2214 punch from 0x27d0b8a to 0x27dd579, (0xc9ef bytes) 2024-09-12T07:20:18.292 INFO:teuthology.orchestra.run.smithi046.stdout:2215 writesame 0x97ab3fe thru 0x97abf27 data_size 0x1(0xb2a bytes) 2024-09-12T07:20:18.298 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:18.299 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:18.299 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:18.299 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:18.301 INFO:teuthology.orchestra.run.smithi046.stdout:2217 trunc from 0xa9791c5 to 0xed01400 2024-09-12T07:20:18.311 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:18.311 INFO:teuthology.orchestra.run.smithi046.stdout:2218 trunc from 0xed01400 to 0xbc31706 2024-09-12T07:20:18.315 INFO:teuthology.orchestra.run.smithi046.stdout:2220 write 0x9d6067d thru 0x9d67a17 (0x739b bytes) 2024-09-12T07:20:18.318 INFO:teuthology.orchestra.run.smithi046.stdout:2224 writesame 0x32dee67 thru 0x32e51dc data_size 0x2(0x6376 bytes) 2024-09-12T07:20:18.326 INFO:teuthology.orchestra.run.smithi046.stdout:2225 read 0x3c0d85e thru 0x3c189dc (0xb17f bytes) 2024-09-12T07:20:18.326 INFO:teuthology.orchestra.run.smithi046.stdout:2226 read 0x82c84e4 thru 0x82d4b86 (0xc6a3 bytes) 2024-09-12T07:20:18.326 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.326 INFO:teuthology.orchestra.run.smithi046.stdout:2228 read 0x7df2af3 thru 0x7dfd8c1 (0xadcf bytes) 2024-09-12T07:20:18.327 INFO:teuthology.orchestra.run.smithi046.stdout:2229 punch from 0x50961a8 to 0x50a0f54, (0xadac bytes) 2024-09-12T07:20:18.327 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.329 INFO:teuthology.orchestra.run.smithi046.stdout:2233 trunc from 0xbc31706 to 0x890affe 2024-09-12T07:20:18.339 INFO:teuthology.orchestra.run.smithi046.stdout:2234 write 0x313c456 thru 0x31406df (0x428a bytes) 2024-09-12T07:20:18.341 INFO:teuthology.orchestra.run.smithi046.stdout:2237 read 0x2573e8c thru 0x2578d7d (0x4ef2 bytes) 2024-09-12T07:20:18.342 INFO:teuthology.orchestra.run.smithi046.stdout:2238 writesame 0x45bcbf9 thru 0x45c5dbf data_size 0x1(0x91c7 bytes) 2024-09-12T07:20:18.361 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:18.362+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:18.367 INFO:teuthology.orchestra.run.smithi046.stdout:2241 writesame 0xc152302 thru 0xc1552f9 data_size 0x14(0x2ff8 bytes) 2024-09-12T07:20:18.367 INFO:teuthology.orchestra.run.smithi046.stdout:2242 write 0xb9df12c thru 0xb9ecf2f (0xde04 bytes) 2024-09-12T07:20:18.370 INFO:teuthology.orchestra.run.smithi046.stdout:2244 read 0x89b918 thru 0x89cbf8 (0x12e1 bytes) 2024-09-12T07:20:18.370 INFO:teuthology.orchestra.run.smithi046.stdout:2245 read 0x4a69f53 thru 0x4a6bb28 (0x1bd6 bytes) 2024-09-12T07:20:18.373 INFO:teuthology.orchestra.run.smithi046.stdout:2246 trunc from 0xc1552fa to 0xa3d0bba 2024-09-12T07:20:18.382 INFO:teuthology.orchestra.run.smithi046.stdout:2249 write 0x41bb8b5 thru 0x41c9b09 (0xe255 bytes) 2024-09-12T07:20:18.391 INFO:teuthology.orchestra.run.smithi046.stdout:2250 write 0xbecc797 thru 0xbed40a0 (0x790a bytes) 2024-09-12T07:20:18.394 INFO:teuthology.orchestra.run.smithi046.stdout:2251 read 0x3c33829 thru 0x3c3d0f1 (0x98c9 bytes) 2024-09-12T07:20:18.394 INFO:teuthology.orchestra.run.smithi046.stdout:2253 write 0xa80016 thru 0xa8d9ce (0xd9b9 bytes) 2024-09-12T07:20:18.396 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.396 INFO:teuthology.orchestra.run.smithi046.stdout:2255 punch from 0x3bf631a to 0x3bf81ba, (0x1ea0 bytes) 2024-09-12T07:20:18.396 INFO:teuthology.orchestra.run.smithi046.stdout:2258 write 0x7e13a99 thru 0x7e1deb4 (0xa41c bytes) 2024-09-12T07:20:18.399 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.399 INFO:teuthology.orchestra.run.smithi046.stdout:2260 read 0x36d4ee5 thru 0x36d5154 (0x270 bytes) 2024-09-12T07:20:18.406 INFO:teuthology.orchestra.run.smithi046.stdout:2261 write 0xd24882a thru 0xd25473e (0xbf15 bytes) 2024-09-12T07:20:18.409 INFO:teuthology.orchestra.run.smithi046.stdout:2263 write 0xa94d9fd thru 0xa95466a (0x6c6e bytes) 2024-09-12T07:20:18.420 INFO:teuthology.orchestra.run.smithi046.stdout:2266 trunc from 0xd25473f to 0x59043a3 2024-09-12T07:20:18.436 INFO:teuthology.orchestra.run.smithi046.stdout:2267 trunc from 0x59043a3 to 0x28286b1 2024-09-12T07:20:18.446 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:18.446 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:18.446 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:18.446 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:18.446 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:18.448 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.448 INFO:teuthology.orchestra.run.smithi046.stdout:2269 read 0x1464ac3 thru 0x1467cbb (0x31f9 bytes) 2024-09-12T07:20:18.448 INFO:teuthology.orchestra.run.smithi046.stdout:2270 read 0x141e5d2 thru 0x142c5c9 (0xdff8 bytes) 2024-09-12T07:20:18.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:18.463 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:18.464+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:18.464 INFO:teuthology.orchestra.run.smithi046.stdout:2271 write 0x966e828 thru 0x9670821 (0x1ffa bytes) 2024-09-12T07:20:18.466 INFO:teuthology.orchestra.run.smithi046.stdout:2272 read 0x3d7227f thru 0x3d75cdc (0x3a5e bytes) 2024-09-12T07:20:18.466 INFO:teuthology.orchestra.run.smithi046.stdout:2275 read 0x841eb57 thru 0x84241d8 (0x5682 bytes) 2024-09-12T07:20:18.466 INFO:teuthology.orchestra.run.smithi046.stdout:2276 read 0x17193a8 thru 0x1725be8 (0xc841 bytes) 2024-09-12T07:20:18.466 INFO:teuthology.orchestra.run.smithi046.stdout:2277 read 0x630184f thru 0x630304b (0x17fd bytes) 2024-09-12T07:20:18.466 INFO:teuthology.orchestra.run.smithi046.stdout:2278 read 0x490388b thru 0x4911080 (0xd7f6 bytes) 2024-09-12T07:20:18.466 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.466 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.474 INFO:teuthology.orchestra.run.smithi046.stdout:2281 write 0xb31c324 thru 0xb3266fe (0xa3db bytes) 2024-09-12T07:20:18.477 INFO:teuthology.orchestra.run.smithi046.stdout:2282 write 0x9f19946 thru 0x9f21ce1 (0x839c bytes) 2024-09-12T07:20:18.480 INFO:teuthology.orchestra.run.smithi046.stdout:2283 writesame 0x38efa0e thru 0x38f8127 data_size 0x1(0x871a bytes) 2024-09-12T07:20:18.501 INFO:teuthology.orchestra.run.smithi046.stdout:2284 trunc from 0xb3266ff to 0xc9109a7 2024-09-12T07:20:18.515 INFO:teuthology.orchestra.run.smithi046.stdout:2286 writesame 0xdad220e thru 0xdadfac7 data_size 0x15(0xd8ba bytes) 2024-09-12T07:20:18.515 INFO:teuthology.orchestra.run.smithi046.stdout:2287 trunc from 0xdadfac8 to 0xe1f1f19 2024-09-12T07:20:18.521 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.529 INFO:teuthology.orchestra.run.smithi046.stdout:2290 trunc from 0xe1f1f19 to 0x6f874f4 2024-09-12T07:20:18.539 INFO:teuthology.orchestra.run.smithi046.stdout:2292 write 0x9459b8 thru 0x945a45 (0x8e bytes) 2024-09-12T07:20:18.540 INFO:teuthology.orchestra.run.smithi046.stdout:2293 read 0x3570e1f thru 0x357d75d (0xc93f bytes) 2024-09-12T07:20:18.541 INFO:teuthology.orchestra.run.smithi046.stdout:2294 read 0x4f6c3d5 thru 0x4f71842 (0x546e bytes) 2024-09-12T07:20:18.541 INFO:teuthology.orchestra.run.smithi046.stdout:2296 read 0x5db34f6 thru 0x5db63f0 (0x2efb bytes) 2024-09-12T07:20:18.547 INFO:teuthology.orchestra.run.smithi046.stdout:2297 trunc from 0x6f874f4 to 0xc9dc8ee 2024-09-12T07:20:18.553 INFO:teuthology.orchestra.run.smithi046.stdout:2299 write 0x750ddfd thru 0x751577b (0x797f bytes) 2024-09-12T07:20:18.556 INFO:teuthology.orchestra.run.smithi046.stdout:2300 punch from 0x628ab34 to 0x6294c15, (0xa0e1 bytes) 2024-09-12T07:20:18.560 INFO:teuthology.orchestra.run.smithi046.stdout:2302 trunc from 0xc9dc8ee to 0x9744257 2024-09-12T07:20:18.563 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:18.564+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:18.569 INFO:teuthology.orchestra.run.smithi046.stdout:2303 trunc from 0x9744257 to 0xce453c8 2024-09-12T07:20:18.575 INFO:teuthology.orchestra.run.smithi046.stdout:2304 punch from 0x1bcda83 to 0x1bd63f6, (0x8973 bytes) 2024-09-12T07:20:18.583 INFO:teuthology.orchestra.run.smithi046.stdout:2306 write 0xecdd4f8 thru 0xecde62d (0x1136 bytes) 2024-09-12T07:20:18.587 INFO:teuthology.orchestra.run.smithi046.stdout:2307 write 0xe47901c thru 0xe47c496 (0x347b bytes) 2024-09-12T07:20:18.589 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:18.589 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:18.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:18.589 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:18.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.589 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77440 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13156352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.590 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:18.590 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.343997098999999, 2024-09-12T07:20:18.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051399724000000001, 2024-09-12T07:20:18.590 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.590 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.590 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77440 2024-09-12T07:20:18.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.591 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:18.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:18.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246267+0000", 2024-09-12T07:20:18.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082256199999999999 2024-09-12T07:20:18.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246271+0000", 2024-09-12T07:20:18.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6700000000000002e-06 2024-09-12T07:20:18.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246283+0000", 2024-09-12T07:20:18.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.207e-05 2024-09-12T07:20:18.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246291+0000", 2024-09-12T07:20:18.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-12T07:20:18.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296645+0000", 2024-09-12T07:20:18.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050353872000000001 2024-09-12T07:20:18.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296729+0000", 2024-09-12T07:20:18.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4215000000000006e-05 2024-09-12T07:20:18.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296844+0000", 2024-09-12T07:20:18.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114626 2024-09-12T07:20:18.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.597 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.598 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77446 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.599 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:18.599 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.342998584, 2024-09-12T07:20:18.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051543643, 2024-09-12T07:20:18.599 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.599 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77446 2024-09-12T07:20:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246579+0000", 2024-09-12T07:20:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013651399999999999 2024-09-12T07:20:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246583+0000", 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1509999999999997e-06 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246593+0000", 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5670000000000001e-06 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246599+0000", 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5189999999999998e-06 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297826+0000", 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051226381000000001 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297903+0000", 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6710000000000002e-05 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297986+0000", 2024-09-12T07:20:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3801000000000006e-05 2024-09-12T07:20:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77447 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 5783552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.34283097, 2024-09-12T07:20:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051512637, 2024-09-12T07:20:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77447 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246812+0000", 2024-09-12T07:20:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000201506 2024-09-12T07:20:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246817+0000", 2024-09-12T07:20:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246828+0000", 2024-09-12T07:20:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1693e-05 2024-09-12T07:20:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246836+0000", 2024-09-12T07:20:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4429999999999997e-06 2024-09-12T07:20:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298021+0000", 2024-09-12T07:20:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051184940999999998 2024-09-12T07:20:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298067+0000", 2024-09-12T07:20:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6292000000000003e-05 2024-09-12T07:20:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298123+0000", 2024-09-12T07:20:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5828999999999997e-05 2024-09-12T07:20:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77448 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.342591232, 2024-09-12T07:20:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051506024999999997, 2024-09-12T07:20:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77448 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246892+0000", 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1854999999999998e-05 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246895+0000", 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3129999999999999e-06 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246904+0000", 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6179999999999995e-06 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246911+0000", 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7939999999999997e-06 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298164+0000", 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051253559999999997 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298249+0000", 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4549000000000007e-05 2024-09-12T07:20:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298356+0000", 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107336 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77449 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.342520036, 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051582830000000003, 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77449 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247137+0000", 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021599499999999999 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247142+0000", 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247154+0000", 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1878e-05 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247168+0000", 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4114000000000001e-05 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298391+0000", 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051223249999999998 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298446+0000", 2024-09-12T07:20:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4366000000000001e-05 2024-09-12T07:20:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298504+0000", 2024-09-12T07:20:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8273000000000003e-05 2024-09-12T07:20:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77450 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12681216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.342260466000001, 2024-09-12T07:20:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051562375000000001, 2024-09-12T07:20:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77450 2024-09-12T07:20:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2799999999999996e-07 2024-09-12T07:20:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247184+0000", 2024-09-12T07:20:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247192+0000", 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5920000000000003e-06 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247197+0000", 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6720000000000001e-06 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298550+0000", 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051352082 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298641+0000", 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1236000000000005e-05 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298743+0000", 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102428 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77451 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7061504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.342236352, 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051813773, 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77451 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247335+0000", 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013012599999999999 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247340+0000", 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.544e-06 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247351+0000", 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.111e-05 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247358+0000", 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9859999999999997e-06 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298909+0000", 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051551781999999997 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298960+0000", 2024-09-12T07:20:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0594999999999999e-05 2024-09-12T07:20:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299019+0000", 2024-09-12T07:20:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8629999999999999e-05 2024-09-12T07:20:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77452 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.342071572, 2024-09-12T07:20:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051892134999999999, 2024-09-12T07:20:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77452 2024-09-12T07:20:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247535+0000", 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165508 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247552+0000", 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6514000000000001e-05 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247565+0000", 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3372e-05 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247577+0000", 2024-09-12T07:20:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:20:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299064+0000", 2024-09-12T07:20:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051486697999999997 2024-09-12T07:20:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299150+0000", 2024-09-12T07:20:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6549000000000001e-05 2024-09-12T07:20:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299262+0000", 2024-09-12T07:20:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011135 2024-09-12T07:20:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77453 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.341850851, 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051901623000000001, 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77453 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247710+0000", 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011951 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247713+0000", 2024-09-12T07:20:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4539999999999999e-06 2024-09-12T07:20:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247720+0000", 2024-09-12T07:20:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0339999999999999e-06 2024-09-12T07:20:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247730+0000", 2024-09-12T07:20:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0030000000000001e-05 2024-09-12T07:20:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299297+0000", 2024-09-12T07:20:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051566446000000002 2024-09-12T07:20:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299382+0000", 2024-09-12T07:20:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4963000000000006e-05 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299492+0000", 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011018600000000001 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77454 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7618560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.341702304, 2024-09-12T07:20:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051921851999999998, 2024-09-12T07:20:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77454 2024-09-12T07:20:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247900+0000", 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016088699999999999 2024-09-12T07:20:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247904+0000", 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6950000000000002e-06 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247912+0000", 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7129999999999994e-06 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247921+0000", 2024-09-12T07:20:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0299999999999999e-06 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299526+0000", 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051604744000000001 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299601+0000", 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5129000000000005e-05 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299661+0000", 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9654000000000002e-05 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77455 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12746752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.341508436, 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051971254000000001, 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77455 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248181+0000", 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024790000000000001 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248185+0000", 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5079999999999999e-06 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248197+0000", 2024-09-12T07:20:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.132e-05 2024-09-12T07:20:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248203+0000", 2024-09-12T07:20:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6320000000000002e-06 2024-09-12T07:20:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299705+0000", 2024-09-12T07:20:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051501342999999998 2024-09-12T07:20:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299799+0000", 2024-09-12T07:20:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4340999999999997e-05 2024-09-12T07:20:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299904+0000", 2024-09-12T07:20:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010521 2024-09-12T07:20:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77456 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7979008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.341226211, 2024-09-12T07:20:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051860577999999997, 2024-09-12T07:20:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77456 2024-09-12T07:20:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248545+0000", 2024-09-12T07:20:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033009700000000001 2024-09-12T07:20:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248550+0000", 2024-09-12T07:20:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248560+0000", 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248568+0000", 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4939999999999997e-06 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299973+0000", 2024-09-12T07:20:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051405470000000002 2024-09-12T07:20:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300019+0000", 2024-09-12T07:20:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5448999999999998e-05 2024-09-12T07:20:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300076+0000", 2024-09-12T07:20:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7073000000000001e-05 2024-09-12T07:20:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77457 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13172736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.340859883, 2024-09-12T07:20:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051737914000000003, 2024-09-12T07:20:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77457 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248582+0000", 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248584+0000", 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248593+0000", 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248609+0000", 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5664000000000001e-05 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300117+0000", 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051507605999999997 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300212+0000", 2024-09-12T07:20:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4919000000000004e-05 2024-09-12T07:20:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300319+0000", 2024-09-12T07:20:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010776399999999999 2024-09-12T07:20:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77458 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.340817465000001, 2024-09-12T07:20:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051863578, 2024-09-12T07:20:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77458 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248627+0000", 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1389999999999999e-06 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248635+0000", 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4019999999999996e-06 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248641+0000", 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1689999999999997e-06 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300359+0000", 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051717889000000003 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300418+0000", 2024-09-12T07:20:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8689e-05 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300487+0000", 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9756000000000001e-05 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77459 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13238272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.340791771999999, 2024-09-12T07:20:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052076504000000003, 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77459 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250611+0000", 2024-09-12T07:20:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019617879999999999 2024-09-12T07:20:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250618+0000", 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1829999999999997e-06 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250629+0000", 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0438000000000001e-05 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250648+0000", 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9355e-05 2024-09-12T07:20:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300528+0000", 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049879252999999998 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300619+0000", 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1910000000000006e-05 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300726+0000", 2024-09-12T07:20:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010657700000000001 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77467 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.338585953000001, 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05160754, 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77467 2024-09-12T07:20:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250856+0000", 2024-09-12T07:20:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:20:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250858+0000", 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9180000000000001e-06 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250864+0000", 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.066e-06 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250869+0000", 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-12T07:20:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302283+0000", 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051414591000000003 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302378+0000", 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4283000000000005e-05 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302463+0000", 2024-09-12T07:20:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5249999999999999e-05 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77468 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14909440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.33856638, 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051742162000000001, 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77468 2024-09-12T07:20:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:20:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250877+0000", 2024-09-12T07:20:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.815e-06 2024-09-12T07:20:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250883+0000", 2024-09-12T07:20:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:20:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250888+0000", 2024-09-12T07:20:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302496+0000", 2024-09-12T07:20:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051607394000000001 2024-09-12T07:20:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302553+0000", 2024-09-12T07:20:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6971000000000001e-05 2024-09-12T07:20:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302617+0000", 2024-09-12T07:20:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4418999999999994e-05 2024-09-12T07:20:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77469 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14974976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.338546201, 2024-09-12T07:20:18.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965694, 2024-09-12T07:20:18.685 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.686 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.686 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77469 2024-09-12T07:20:18.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.686 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:18.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:18.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:18.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:20:18.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250897+0000", 2024-09-12T07:20:18.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9130000000000001e-06 2024-09-12T07:20:18.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250903+0000", 2024-09-12T07:20:18.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0129999999999999e-06 2024-09-12T07:20:18.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250908+0000", 2024-09-12T07:20:18.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9679999999999997e-06 2024-09-12T07:20:18.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302661+0000", 2024-09-12T07:20:18.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051753066 2024-09-12T07:20:18.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302755+0000", 2024-09-12T07:20:18.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3376999999999997e-05 2024-09-12T07:20:18.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302861+0000", 2024-09-12T07:20:18.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105924 2024-09-12T07:20:18.693 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.693 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.694 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77470 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12828672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.694 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:18.694 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.338526524000001, 2024-09-12T07:20:18.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052098589000000001, 2024-09-12T07:20:18.695 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.696 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77470 2024-09-12T07:20:18.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.696 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:18.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:18.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:18.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:18.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250917+0000", 2024-09-12T07:20:18.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8190000000000001e-06 2024-09-12T07:20:18.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250923+0000", 2024-09-12T07:20:18.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:20:18.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250928+0000", 2024-09-12T07:20:18.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9420000000000003e-06 2024-09-12T07:20:18.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302894+0000", 2024-09-12T07:20:18.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965863000000001 2024-09-12T07:20:18.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302955+0000", 2024-09-12T07:20:18.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1235999999999994e-05 2024-09-12T07:20:18.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303013+0000", 2024-09-12T07:20:18.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8167000000000001e-05 2024-09-12T07:20:18.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.704 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.705 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77471 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12894208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:18.705 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:18.705 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.338506978, 2024-09-12T07:20:18.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052326263999999997, 2024-09-12T07:20:18.705 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:18.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:18.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:18.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:18.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:18.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:18.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77471 2024-09-12T07:20:18.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.707 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:18.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:18.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:18.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:18.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:18.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:18.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:18.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256961+0000", 2024-09-12T07:20:18.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060271209999999999 2024-09-12T07:20:18.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:18.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256972+0000", 2024-09-12T07:20:18.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0755e-05 2024-09-12T07:20:18.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:18.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256986+0000", 2024-09-12T07:20:18.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845e-05 2024-09-12T07:20:18.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:18.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256996+0000", 2024-09-12T07:20:18.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:18.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303060+0000", 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.046063661999999998 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303149+0000", 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8949999999999994e-05 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303261+0000", 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011195 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.714 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:18.715 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:18.715 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:18.716 INFO:teuthology.orchestra.run.smithi046.stdout:2308 punch from 0xd6176a0 to 0xd61945b, (0x1dbb bytes) 2024-09-12T07:20:18.716 INFO:teuthology.orchestra.run.smithi046.stdout:2309 read 0xd791193 thru 0xd79d75d (0xc5cb bytes) 2024-09-12T07:20:18.716 INFO:teuthology.orchestra.run.smithi046.stdout:2312 punch from 0xec78da7 to 0xec7ec99, (0x5ef2 bytes) 2024-09-12T07:20:18.716 INFO:teuthology.orchestra.run.smithi046.stdout:2313 read 0x6268d06 thru 0x626f8ea (0x6be5 bytes) 2024-09-12T07:20:18.716 INFO:teuthology.orchestra.run.smithi046.stdout:2315 write 0x5f2eea0 thru 0x5f3e06d (0xf1ce bytes) 2024-09-12T07:20:18.717 INFO:teuthology.orchestra.run.smithi046.stdout:2316 read 0xe78312d thru 0xe789326 (0x61fa bytes) 2024-09-12T07:20:18.717 INFO:teuthology.orchestra.run.smithi046.stdout:2317 punch from 0xa3efbf5 to 0xa3efdc1, (0x1cc bytes) 2024-09-12T07:20:18.717 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:18.721 INFO:teuthology.orchestra.run.smithi046.stdout:2319 trunc from 0xecde62e to 0x1ba5d89 2024-09-12T07:20:18.721 INFO:teuthology.orchestra.run.smithi046.stdout:2320 punch from 0xf3352b to 0xf3650e, (0x2fe3 bytes) 2024-09-12T07:20:18.721 INFO:teuthology.orchestra.run.smithi046.stdout:2321 writesame 0x4aea2cd thru 0x4af65b9 data_size 0x1(0xc2ed bytes) 2024-09-12T07:20:18.721 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.721 INFO:teuthology.orchestra.run.smithi046.stdout:2324 read 0x21d8e16 thru 0x21e1214 (0x83ff bytes) 2024-09-12T07:20:18.721 INFO:teuthology.orchestra.run.smithi046.stdout:2326 trunc from 0x4af65ba to 0x45cccca 2024-09-12T07:20:18.721 INFO:teuthology.orchestra.run.smithi046.stdout:2327 write 0xaa2d056 thru 0xaa31678 (0x4623 bytes) 2024-09-12T07:20:18.721 INFO:teuthology.orchestra.run.smithi046.stdout:2328 trunc from 0xaa31679 to 0xe3b5d35 2024-09-12T07:20:18.721 INFO:teuthology.orchestra.run.smithi046.stdout:2329 punch from 0xb4e8871 to 0xb4eadcc, (0x255b bytes) 2024-09-12T07:20:18.721 INFO:teuthology.orchestra.run.smithi046.stdout:2330 writesame 0x9d99503 thru 0x9da463c data_size 0x41(0xb13a bytes) 2024-09-12T07:20:18.721 INFO:teuthology.orchestra.run.smithi046.stdout:2331 write 0x226435f thru 0x226db69 (0x980b bytes) 2024-09-12T07:20:18.721 INFO:teuthology.orchestra.run.smithi046.stdout:2332 punch from 0x9c330da to 0x9c3e6df, (0xb605 bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2334 read 0xba833fd thru 0xba91ad6 (0xe6da bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2335 punch from 0xb50ab64 to 0xb511331, (0x67cd bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2336 writesame 0x15ba6d6 thru 0x15c0155 data_size 0x40(0x5a80 bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2337 writesame 0x5449694 thru 0x544d261 data_size 0x5(0x3bce bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2338 write 0x9728b7 thru 0x97dfde (0xb728 bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2339 write 0xd4385e9 thru 0xd43cb4a (0x4562 bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2340 write 0x4fcd890 thru 0x4fd624b (0x89bc bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2341 punch from 0xc6c0f71 to 0xc6c2af5, (0x1b84 bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2343 writesame 0x7b6ca6b thru 0x7b745c4 data_size 0x39(0x7b5a bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2344 read 0x7f74ec2 thru 0x7f8146a (0xc5a9 bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2345 write 0xa6f0c79 thru 0xa6f16e0 (0xa68 bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2346 trunc from 0xe3b5d35 to 0x5bf96e3 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2347 read 0x336d047 thru 0x33789f5 (0xb9af bytes) 2024-09-12T07:20:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:2348 punch from 0x31c3a29 to 0x31c462e, (0xc05 bytes) 2024-09-12T07:20:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:2349 read 0x140d492 thru 0x14137ab (0x631a bytes) 2024-09-12T07:20:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:2351 read 0x4392251 thru 0x43951fd (0x2fad bytes) 2024-09-12T07:20:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:2352 writesame 0x4dd31de thru 0x4dd6205 data_size 0x43(0x3028 bytes) 2024-09-12T07:20:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:2353 read 0x29b30d9 thru 0x29c07ab (0xd6d3 bytes) 2024-09-12T07:20:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:2356 punch from 0x2e448c5 to 0x2e47e84, (0x35bf bytes) 2024-09-12T07:20:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:2357 read 0x4a2a43b thru 0x4a36d16 (0xc8dc bytes) 2024-09-12T07:20:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:2358 read 0x57eb2dc thru 0x57edffc (0x2d21 bytes) 2024-09-12T07:20:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:2360 read 0x39b026f thru 0x39b8148 (0x7eda bytes) 2024-09-12T07:20:18.724 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:18.716+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:18.724 INFO:teuthology.orchestra.run.smithi046.stdout:2361 write 0xd96f0a9 thru 0xd971f50 (0x2ea8 bytes) 2024-09-12T07:20:18.724 INFO:teuthology.orchestra.run.smithi046.stdout:2362 read 0x70f1a62 thru 0x70fb7c3 (0x9d62 bytes) 2024-09-12T07:20:18.726 INFO:teuthology.orchestra.run.smithi046.stdout:2363 write 0x9e576e2 thru 0x9e628cc (0xb1eb bytes) 2024-09-12T07:20:18.727 INFO:teuthology.orchestra.run.smithi046.stdout:2364 write 0x602d213 thru 0x6038139 (0xaf27 bytes) 2024-09-12T07:20:18.730 INFO:teuthology.orchestra.run.smithi046.stdout:2365 write 0x59ae834 thru 0x59b0935 (0x2102 bytes) 2024-09-12T07:20:18.733 INFO:teuthology.orchestra.run.smithi046.stdout:2366 read 0x32917c9 thru 0x32980a4 (0x68dc bytes) 2024-09-12T07:20:18.733 INFO:teuthology.orchestra.run.smithi046.stdout:2367 read 0x54a3f72 thru 0x54abbaa (0x7c39 bytes) 2024-09-12T07:20:18.733 INFO:teuthology.orchestra.run.smithi046.stdout:2368 clone 17 order 18 su 131072 sc 8 2024-09-12T07:20:18.817 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:18.818+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:18.832 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:18.833 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:18.833 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:18.833 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:18.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:20:18.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:18.917 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:18.918+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:18.970 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:18.987 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:18.987 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:18.999 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:18.999 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:18.999 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:18.999 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:19.000 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:19.004 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:19.005 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:19.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:19.018 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:19.019+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:19.021 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:19.021 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:19.045 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:19.045 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:19.062 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:19.062 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:19.079 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:19.096 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:19.097 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:19.118 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:19.119+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:19.120 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:19.120 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:19.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:19.220 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:19.222+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:19.241 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:19.241 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:19.241 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.3 to 1 (10000) 2024-09-12T07:20:19.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T07:20:19.251 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:19.258 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-12T07:20:19.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:20:19.314 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone16 from 0xd971f51 (overlap 0x5b3a5b) to 0x0 2024-09-12T07:20:19.320 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:19.321+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:19.363 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:19.363 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:19.372 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:19.384 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-12T07:20:19.397 INFO:teuthology.orchestra.run.smithi046.stdout:2369 writesame 0xbc9e698 thru 0xbcae621 data_size 0x3(0xff8a bytes) 2024-09-12T07:20:19.414 INFO:teuthology.orchestra.run.smithi046.stdout:2370 read 0x2dde027 thru 0x2de976c (0xb746 bytes) 2024-09-12T07:20:19.420 INFO:teuthology.orchestra.run.smithi046.stdout:2372 trunc from 0xd971f51 to 0x99a0836 2024-09-12T07:20:19.420 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:19.422+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:19.442 INFO:teuthology.orchestra.run.smithi046.stdout:2373 write 0xaf19458 thru 0xaf2772b (0xe2d4 bytes) 2024-09-12T07:20:19.446 INFO:teuthology.orchestra.run.smithi046.stdout:2374 write 0x184ea5e thru 0x1857e7e (0x9421 bytes) 2024-09-12T07:20:19.454 INFO:teuthology.orchestra.run.smithi046.stdout:2376 trunc from 0xaf2772c to 0xe7d284a 2024-09-12T07:20:19.460 INFO:teuthology.orchestra.run.smithi046.stdout:2377 read 0xae08e2f thru 0xae121b3 (0x9385 bytes) 2024-09-12T07:20:19.460 INFO:teuthology.orchestra.run.smithi046.stdout:2378 write 0x3ee9761 thru 0x3eef4aa (0x5d4a bytes) 2024-09-12T07:20:19.464 INFO:teuthology.orchestra.run.smithi046.stdout:2379 punch from 0x97e0f9b to 0x97e4a01, (0x3a66 bytes) 2024-09-12T07:20:19.464 INFO:teuthology.orchestra.run.smithi046.stdout:2380 punch from 0x505f764 to 0x50657b4, (0x6050 bytes) 2024-09-12T07:20:19.464 INFO:teuthology.orchestra.run.smithi046.stdout:2381 read 0x45d936b thru 0x45e6649 (0xd2df bytes) 2024-09-12T07:20:19.465 INFO:teuthology.orchestra.run.smithi046.stdout:2382 read 0xe3efac9 thru 0xe3f3a07 (0x3f3f bytes) 2024-09-12T07:20:19.465 INFO:teuthology.orchestra.run.smithi046.stdout:2385 read 0x6a73b4d thru 0x6a790dd (0x5591 bytes) 2024-09-12T07:20:19.465 INFO:teuthology.orchestra.run.smithi046.stdout:2386 write 0x84ee143 thru 0x84f64dd (0x839b bytes) 2024-09-12T07:20:19.478 INFO:teuthology.orchestra.run.smithi046.stdout:2387 trunc from 0xe7d284a to 0x5d128ec 2024-09-12T07:20:19.486 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:19.486 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:19.496 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:19.510 INFO:teuthology.orchestra.run.smithi046.stdout:2390 write 0x6050745 thru 0x605aac8 (0xa384 bytes) 2024-09-12T07:20:19.513 INFO:teuthology.orchestra.run.smithi046.stdout:2392 read 0x9e3fe9 thru 0x9e5c6f (0x1c87 bytes) 2024-09-12T07:20:19.513 INFO:teuthology.orchestra.run.smithi046.stdout:2394 punch from 0x16cbfbd to 0x16cd354, (0x1397 bytes) 2024-09-12T07:20:19.520 INFO:teuthology.orchestra.run.smithi046.stdout:2396 trunc from 0x605aac9 to 0x4f1471 2024-09-12T07:20:19.539 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:19.540+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:19.565 INFO:teuthology.orchestra.run.smithi046.stdout:2397 writesame 0xe86ca60 thru 0xe879773 data_size 0x96(0xcd14 bytes) 2024-09-12T07:20:19.565 INFO:teuthology.orchestra.run.smithi046.stdout:2398 writesame 0x5362e1 thru 0x539221 data_size 0x1(0x2f41 bytes) 2024-09-12T07:20:19.572 INFO:teuthology.orchestra.run.smithi046.stdout:2401 write 0x7e16f99 thru 0x7e172c7 (0x32f bytes) 2024-09-12T07:20:19.576 INFO:teuthology.orchestra.run.smithi046.stdout:2403 write 0xbac2228 thru 0xbac8c6f (0x6a48 bytes) 2024-09-12T07:20:19.584 INFO:teuthology.orchestra.run.smithi046.stdout:2406 trunc from 0xe879774 to 0xa7701ea 2024-09-12T07:20:19.594 INFO:teuthology.orchestra.run.smithi046.stdout:2407 write 0x4d0f730 thru 0x4d1dfe4 (0xe8b5 bytes) 2024-09-12T07:20:19.598 INFO:teuthology.orchestra.run.smithi046.stdout:2408 write 0x65a0d96 thru 0x65ad6a6 (0xc911 bytes) 2024-09-12T07:20:19.601 INFO:teuthology.orchestra.run.smithi046.stdout:2409 read 0x88b20a thru 0x88f4d1 (0x42c8 bytes) 2024-09-12T07:20:19.601 INFO:teuthology.orchestra.run.smithi046.stdout:2410 read 0x8e37ace thru 0x8e40ef2 (0x9425 bytes) 2024-09-12T07:20:19.601 INFO:teuthology.orchestra.run.smithi046.stdout:2411 read 0x5ed76c9 thru 0x5edfee1 (0x8819 bytes) 2024-09-12T07:20:19.606 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:19.606 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:19.612 INFO:teuthology.orchestra.run.smithi046.stdout:2413 trunc from 0xa7701ea to 0x8301e2 2024-09-12T07:20:19.615 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:19.621 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:19.623+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:19.632 INFO:teuthology.orchestra.run.smithi046.stdout:2414 write 0x3dcd697 thru 0x3ddb14d (0xdab7 bytes) 2024-09-12T07:20:19.650 INFO:teuthology.orchestra.run.smithi046.stdout:2415 write 0xc1eaa5c thru 0xc1f28d0 (0x7e75 bytes) 2024-09-12T07:20:19.654 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:20:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:20:19.655 INFO:teuthology.orchestra.run.smithi046.stdout:2416 writesame 0x379fa45 thru 0x37a51c5 data_size 0x39(0x5781 bytes) 2024-09-12T07:20:19.657 INFO:teuthology.orchestra.run.smithi046.stdout:2417 read 0x3452f15 thru 0x3460d32 (0xde1e bytes) 2024-09-12T07:20:19.670 INFO:teuthology.orchestra.run.smithi046.stdout:2418 trunc from 0xc1f28d1 to 0x672cbf 2024-09-12T07:20:19.673 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-12T07:20:19.499201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":560874233,"num_objects":654,"num_object_clones":134,"num_object_copies":1308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":654,"num_whiteouts":268,"num_read":111210,"num_read_kb":411090,"num_write":5866,"num_write_kb":283650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":49942512,"num_keys_recovered":48,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4796,"ondisk_log_size":4796,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":709352,"kb_used_data":430624,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754265368,"statfs":{"total":773094113280,"available":772367736832,"internally_reserved":0,"allocated":440958976,"data_stored":455370407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":150503677,"num_objects":246,"num_object_clones":47,"num_object_copies":492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":246,"num_whiteouts":153,"num_read":11529,"num_read_kb":92519,"num_write":1798,"num_write_kb":92725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000569"},"pg_stats":[{"pgid":"3.1f","version":"57'242","reported_seq":1844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.232649+0000","last_change":"2024-09-12T07:19:44.455827+0000","last_active":"2024-09-12T07:20:17.232649+0000","last_peered":"2024-09-12T07:20:17.232649+0000","last_clean":"2024-09-12T07:20:17.232649+0000","last_became_active":"2024-09-12T07:19:42.451212+0000","last_became_peered":"2024-09-12T07:19:42.451212+0000","last_unstale":"2024-09-12T07:20:17.232649+0000","last_undegraded":"2024-09-12T07:20:17.232649+0000","last_fullsized":"2024-09-12T07:20:17.232649+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:57.585258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001023672,"stat_sum":{"num_bytes":10940340,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":2844,"num_read_kb":15275,"num_write":311,"num_write_kb":4391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"57'118","reported_seq":141,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.274011+0000","last_change":"2024-09-12T07:20:07.305432+0000","last_active":"2024-09-12T07:20:17.274011+0000","last_peered":"2024-09-12T07:20:17.274011+0000","last_clean":"2024-09-12T07:20:17.274011+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:20:17.274011+0000","last_undegraded":"2024-09-12T07:20:17.274011+0000","last_fullsized":"2024-09-12T07:20:17.274011+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:58.448307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081935099999999996,"stat_sum":{"num_bytes":28822701,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":16696,"num_read_kb":20572,"num_write":107,"num_write_kb":16946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"57'98","reported_seq":17107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.252898+0000","last_change":"2024-09-12T07:19:44.464376+0000","last_active":"2024-09-12T07:20:17.252898+0000","last_peered":"2024-09-12T07:20:17.252898+0000","last_clean":"2024-09-12T07:20:17.252898+0000","last_became_active":"2024-09-12T07:19:42.451973+0000","last_became_peered":"2024-09-12T07:19:42.451973+0000","last_unstale":"2024-09-12T07:20:17.252898+0000","last_undegraded":"2024-09-12T07:20:17.252898+0000","last_fullsized":"2024-09-12T07:20:17.252898+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:27.976836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036949999999999,"stat_sum":{"num_bytes":26782347,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":16959,"num_read_kb":23181,"num_write":89,"num_write_kb":10322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'144","reported_seq":1698,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.358238+0000","last_change":"2024-09-12T07:19:44.464287+0000","last_active":"2024-09-12T07:20:17.358238+0000","last_peered":"2024-09-12T07:20:17.358238+0000","last_clean":"2024-09-12T07:20:17.358238+0000","last_became_active":"2024-09-12T07:19:42.455338+0000","last_became_peered":"2024-09-12T07:19:42.455338+0000","last_unstale":"2024-09-12T07:20:17.358238+0000","last_undegraded":"2024-09-12T07:20:17.358238+0000","last_fullsized":"2024-09-12T07:20:17.358238+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:50.776177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012347180000000001,"stat_sum":{"num_bytes":13823415,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":1857,"num_read_kb":9757,"num_write":182,"num_write_kb":2570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"56'97","reported_seq":12959,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.232468+0000","last_change":"2024-09-12T07:19:44.456652+0000","last_active":"2024-09-12T07:20:17.232468+0000","last_peered":"2024-09-12T07:20:17.232468+0000","last_clean":"2024-09-12T07:20:17.232468+0000","last_became_active":"2024-09-12T07:19:42.454990+0000","last_became_peered":"2024-09-12T07:19:42.454990+0000","last_unstale":"2024-09-12T07:20:17.232468+0000","last_undegraded":"2024-09-12T07:20:17.232468+0000","last_fullsized":"2024-09-12T07:20:17.232468+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:13:53.218197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049866,"stat_sum":{"num_bytes":20900457,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":13242,"num_read_kb":14669,"num_write":88,"num_write_kb":11790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"57'114","reported_seq":1053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.377750+0000","last_change":"2024-09-12T07:19:44.459417+0000","last_active":"2024-09-12T07:20:17.377750+0000","last_peered":"2024-09-12T07:20:17.377750+0000","last_clean":"2024-09-12T07:20:17.377750+0000","last_became_active":"2024-09-12T07:19:42.454620+0000","last_became_peered":"2024-09-12T07:19:42.454620+0000","last_unstale":"2024-09-12T07:20:17.377750+0000","last_undegraded":"2024-09-12T07:20:17.377750+0000","last_fullsized":"2024-09-12T07:20:17.377750+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:06.854794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090501500000000001,"stat_sum":{"num_bytes":13596786,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":880,"num_read_kb":2263,"num_write":108,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"57'243","reported_seq":1072,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.422045+0000","last_change":"2024-09-12T07:20:10.196744+0000","last_active":"2024-09-12T07:20:17.422045+0000","last_peered":"2024-09-12T07:20:17.422045+0000","last_clean":"2024-09-12T07:20:17.422045+0000","last_became_active":"2024-09-12T07:20:10.195312+0000","last_became_peered":"2024-09-12T07:20:10.195312+0000","last_unstale":"2024-09-12T07:20:17.422045+0000","last_undegraded":"2024-09-12T07:20:17.422045+0000","last_fullsized":"2024-09-12T07:20:17.422045+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:13.894288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113156,"stat_sum":{"num_bytes":16795101,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":1503,"num_read_kb":10040,"num_write":354,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1101525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"54'118","reported_seq":282,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.274002+0000","last_change":"2024-09-12T07:20:07.446518+0000","last_active":"2024-09-12T07:20:17.274002+0000","last_peered":"2024-09-12T07:20:17.274002+0000","last_clean":"2024-09-12T07:20:17.274002+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:20:17.274002+0000","last_undegraded":"2024-09-12T07:20:17.274002+0000","last_fullsized":"2024-09-12T07:20:17.274002+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:36:33.763137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000443294,"stat_sum":{"num_bytes":22996397,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":2082,"num_read_kb":16378,"num_write":123,"num_write_kb":5396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"55'60","reported_seq":2447,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.232849+0000","last_change":"2024-09-12T07:19:44.456115+0000","last_active":"2024-09-12T07:20:17.232849+0000","last_peered":"2024-09-12T07:20:17.232849+0000","last_clean":"2024-09-12T07:20:17.232849+0000","last_became_active":"2024-09-12T07:19:42.452617+0000","last_became_peered":"2024-09-12T07:19:42.452617+0000","last_unstale":"2024-09-12T07:20:17.232849+0000","last_undegraded":"2024-09-12T07:20:17.232849+0000","last_fullsized":"2024-09-12T07:20:17.232849+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:25.151293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077251,"stat_sum":{"num_bytes":24337497,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":2312,"num_read_kb":9400,"num_write":63,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"57'59","reported_seq":160,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.317817+0000","last_change":"2024-09-12T07:19:44.463507+0000","last_active":"2024-09-12T07:20:17.317817+0000","last_peered":"2024-09-12T07:20:17.317817+0000","last_clean":"2024-09-12T07:20:17.317817+0000","last_became_active":"2024-09-12T07:19:42.455210+0000","last_became_peered":"2024-09-12T07:19:42.455210+0000","last_unstale":"2024-09-12T07:20:17.317817+0000","last_undegraded":"2024-09-12T07:20:17.317817+0000","last_fullsized":"2024-09-12T07:20:17.317817+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:12.237223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049753700000000004,"stat_sum":{"num_bytes":12309059,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":8,"num_read":25,"num_read_kb":108,"num_write":51,"num_write_kb":4628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"57'86","reported_seq":410,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.309870+0000","last_change":"2024-09-12T07:20:07.261139+0000","last_active":"2024-09-12T07:20:17.309870+0000","last_peered":"2024-09-12T07:20:17.309870+0000","last_clean":"2024-09-12T07:20:17.309870+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:20:17.309870+0000","last_undegraded":"2024-09-12T07:20:17.309870+0000","last_fullsized":"2024-09-12T07:20:17.309870+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:57:02.011265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012225490000000001,"stat_sum":{"num_bytes":7216656,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":487,"num_read_kb":1445,"num_write":72,"num_write_kb":4344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"57'121","reported_seq":3850,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.370365+0000","last_change":"2024-09-12T07:19:44.481796+0000","last_active":"2024-09-12T07:20:17.370365+0000","last_peered":"2024-09-12T07:20:17.370365+0000","last_clean":"2024-09-12T07:20:17.370365+0000","last_became_active":"2024-09-12T07:19:42.452277+0000","last_became_peered":"2024-09-12T07:19:42.452277+0000","last_unstale":"2024-09-12T07:20:17.370365+0000","last_undegraded":"2024-09-12T07:20:17.370365+0000","last_fullsized":"2024-09-12T07:20:17.370365+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:59.618843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012485160000000001,"stat_sum":{"num_bytes":31451428,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":4114,"num_read_kb":49211,"num_write":128,"num_write_kb":23849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.289203+0000","last_change":"2024-09-12T07:19:32.474512+0000","last_active":"2024-09-12T07:20:17.289203+0000","last_peered":"2024-09-12T07:20:17.289203+0000","last_clean":"2024-09-12T07:20:17.289203+0000","last_became_active":"2024-09-12T07:19:30.219990+0000","last_became_peered":"2024-09-12T07:19:30.219990+0000","last_unstale":"2024-09-12T07:20:17.289203+0000","last_undegraded":"2024-09-12T07:20:17.289203+0000","last_fullsized":"2024-09-12T07:20:17.289203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:32.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046339800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"57'261","reported_seq":1203,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.421897+0000","last_change":"2024-09-12T07:19:44.455257+0000","last_active":"2024-09-12T07:20:17.421897+0000","last_peered":"2024-09-12T07:20:17.421897+0000","last_clean":"2024-09-12T07:20:17.421897+0000","last_became_active":"2024-09-12T07:19:42.449730+0000","last_became_peered":"2024-09-12T07:19:42.449730+0000","last_unstale":"2024-09-12T07:20:17.421897+0000","last_undegraded":"2024-09-12T07:20:17.421897+0000","last_fullsized":"2024-09-12T07:20:17.421897+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:18:04.089947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059259600000000003,"stat_sum":{"num_bytes":9633736,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":1792,"num_read_kb":8645,"num_write":308,"num_write_kb":1367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.235223+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:20:17.235223+0000","last_peered":"2024-09-12T07:20:17.235223+0000","last_clean":"2024-09-12T07:20:17.235223+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:20:17.235223+0000","last_undegraded":"2024-09-12T07:20:17.235223+0000","last_fullsized":"2024-09-12T07:20:17.235223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"57'76","reported_seq":685,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.422556+0000","last_change":"2024-09-12T07:20:07.211023+0000","last_active":"2024-09-12T07:20:17.422556+0000","last_peered":"2024-09-12T07:20:17.422556+0000","last_clean":"2024-09-12T07:20:17.422556+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:20:17.422556+0000","last_undegraded":"2024-09-12T07:20:17.422556+0000","last_fullsized":"2024-09-12T07:20:17.422556+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:45.978689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087343100000000003,"stat_sum":{"num_bytes":22331570,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":668,"num_read_kb":11201,"num_write":79,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.242263+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:20:17.242263+0000","last_peered":"2024-09-12T07:20:17.242263+0000","last_clean":"2024-09-12T07:20:17.242263+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:20:17.242263+0000","last_undegraded":"2024-09-12T07:20:17.242263+0000","last_fullsized":"2024-09-12T07:20:17.242263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"55'166","reported_seq":525,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.235066+0000","last_change":"2024-09-12T07:19:44.458925+0000","last_active":"2024-09-12T07:20:17.235066+0000","last_peered":"2024-09-12T07:20:17.235066+0000","last_clean":"2024-09-12T07:20:17.235066+0000","last_became_active":"2024-09-12T07:19:42.452704+0000","last_became_peered":"2024-09-12T07:19:42.452704+0000","last_unstale":"2024-09-12T07:20:17.235066+0000","last_undegraded":"2024-09-12T07:20:17.235066+0000","last_fullsized":"2024-09-12T07:20:17.235066+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:35.846662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045577200000000001,"stat_sum":{"num_bytes":2876510,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":1189,"num_read_kb":1980,"num_write":236,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.234965+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:20:17.234965+0000","last_peered":"2024-09-12T07:20:17.234965+0000","last_clean":"2024-09-12T07:20:17.234965+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:20:17.234965+0000","last_undegraded":"2024-09-12T07:20:17.234965+0000","last_fullsized":"2024-09-12T07:20:17.234965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"57'110","reported_seq":3254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.336522+0000","last_change":"2024-09-12T07:20:07.289858+0000","last_active":"2024-09-12T07:20:17.336522+0000","last_peered":"2024-09-12T07:20:17.336522+0000","last_clean":"2024-09-12T07:20:17.336522+0000","last_became_active":"2024-09-12T07:20:07.183819+0000","last_became_peered":"2024-09-12T07:20:07.183819+0000","last_unstale":"2024-09-12T07:20:17.336522+0000","last_undegraded":"2024-09-12T07:20:17.336522+0000","last_fullsized":"2024-09-12T07:20:17.336522+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:25:17.468030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015134619999999999,"stat_sum":{"num_bytes":44012764,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":3059,"num_read_kb":23450,"num_write":139,"num_write_kb":17370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":3070354,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"57'63","reported_seq":580,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.307609+0000","last_change":"2024-09-12T07:19:44.456465+0000","last_active":"2024-09-12T07:20:17.307609+0000","last_peered":"2024-09-12T07:20:17.307609+0000","last_clean":"2024-09-12T07:20:17.307609+0000","last_became_active":"2024-09-12T07:19:42.454103+0000","last_became_peered":"2024-09-12T07:19:42.454103+0000","last_unstale":"2024-09-12T07:20:17.307609+0000","last_undegraded":"2024-09-12T07:20:17.307609+0000","last_fullsized":"2024-09-12T07:20:17.307609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:11.047251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010980880000000001,"stat_sum":{"num_bytes":7410324,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":14,"num_read":441,"num_read_kb":1640,"num_write":47,"num_write_kb":1264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.252568+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:20:17.252568+0000","last_peered":"2024-09-12T07:20:17.252568+0000","last_clean":"2024-09-12T07:20:17.252568+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:20:17.252568+0000","last_undegraded":"2024-09-12T07:20:17.252568+0000","last_fullsized":"2024-09-12T07:20:17.252568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"53'135","reported_seq":818,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.235066+0000","last_change":"2024-09-12T07:19:44.458868+0000","last_active":"2024-09-12T07:20:17.235066+0000","last_peered":"2024-09-12T07:20:17.235066+0000","last_clean":"2024-09-12T07:20:17.235066+0000","last_became_active":"2024-09-12T07:19:42.451807+0000","last_became_peered":"2024-09-12T07:19:42.451807+0000","last_unstale":"2024-09-12T07:20:17.235066+0000","last_undegraded":"2024-09-12T07:20:17.235066+0000","last_fullsized":"2024-09-12T07:20:17.235066+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.743040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048363399999999998,"stat_sum":{"num_bytes":12607390,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":1548,"num_read_kb":3970,"num_write":144,"num_write_kb":8640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.229409+0000","last_change":"2024-09-12T07:20:07.183543+0000","last_active":"2024-09-12T07:20:17.229409+0000","last_peered":"2024-09-12T07:20:17.229409+0000","last_clean":"2024-09-12T07:20:17.229409+0000","last_became_active":"2024-09-12T07:20:07.183213+0000","last_became_peered":"2024-09-12T07:20:07.183213+0000","last_unstale":"2024-09-12T07:20:17.229409+0000","last_undegraded":"2024-09-12T07:20:17.229409+0000","last_fullsized":"2024-09-12T07:20:17.229409+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.634459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"57'93","reported_seq":1073,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.328840+0000","last_change":"2024-09-12T07:19:44.455560+0000","last_active":"2024-09-12T07:20:17.328840+0000","last_peered":"2024-09-12T07:20:17.328840+0000","last_clean":"2024-09-12T07:20:17.328840+0000","last_became_active":"2024-09-12T07:19:42.451970+0000","last_became_peered":"2024-09-12T07:19:42.451970+0000","last_unstale":"2024-09-12T07:20:17.328840+0000","last_undegraded":"2024-09-12T07:20:17.328840+0000","last_fullsized":"2024-09-12T07:20:17.328840+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:26.326667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083352100000000004,"stat_sum":{"num_bytes":18618824,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":1836,"num_read_kb":10738,"num_write":98,"num_write_kb":15485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.241930+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:20:17.241930+0000","last_peered":"2024-09-12T07:20:17.241930+0000","last_clean":"2024-09-12T07:20:17.241930+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:20:17.241930+0000","last_undegraded":"2024-09-12T07:20:17.241930+0000","last_fullsized":"2024-09-12T07:20:17.241930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"57'623","reported_seq":1113,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.374159+0000","last_change":"2024-09-12T07:20:07.467030+0000","last_active":"2024-09-12T07:20:17.374159+0000","last_peered":"2024-09-12T07:20:17.374159+0000","last_clean":"2024-09-12T07:20:17.374159+0000","last_became_active":"2024-09-12T07:20:07.180577+0000","last_became_peered":"2024-09-12T07:20:07.180577+0000","last_unstale":"2024-09-12T07:20:17.374159+0000","last_undegraded":"2024-09-12T07:20:17.374159+0000","last_fullsized":"2024-09-12T07:20:17.374159+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:47.344526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089638599999999999,"stat_sum":{"num_bytes":15519005,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":3427,"num_read_kb":9930,"num_write":1046,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":261251,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":91,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.252567+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:20:17.252567+0000","last_peered":"2024-09-12T07:20:17.252567+0000","last_clean":"2024-09-12T07:20:17.252567+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:20:17.252567+0000","last_undegraded":"2024-09-12T07:20:17.252567+0000","last_fullsized":"2024-09-12T07:20:17.252567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":181,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.241930+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:20:17.241930+0000","last_peered":"2024-09-12T07:20:17.241930+0000","last_clean":"2024-09-12T07:20:17.241930+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:20:17.241930+0000","last_undegraded":"2024-09-12T07:20:17.241930+0000","last_fullsized":"2024-09-12T07:20:17.241930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"57'114","reported_seq":503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.310049+0000","last_change":"2024-09-12T07:20:07.428482+0000","last_active":"2024-09-12T07:20:17.310049+0000","last_peered":"2024-09-12T07:20:17.310049+0000","last_clean":"2024-09-12T07:20:17.310049+0000","last_became_active":"2024-09-12T07:20:07.180763+0000","last_became_peered":"2024-09-12T07:20:07.180763+0000","last_unstale":"2024-09-12T07:20:17.310049+0000","last_undegraded":"2024-09-12T07:20:17.310049+0000","last_fullsized":"2024-09-12T07:20:17.310049+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:33.693067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047491800000000001,"stat_sum":{"num_bytes":23252826,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":7,"num_read":604,"num_read_kb":8717,"num_write":154,"num_write_kb":14586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":8633990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"57'235","reported_seq":3231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.425624+0000","last_change":"2024-09-12T07:19:44.469508+0000","last_active":"2024-09-12T07:20:17.425624+0000","last_peered":"2024-09-12T07:20:17.425624+0000","last_clean":"2024-09-12T07:20:17.425624+0000","last_became_active":"2024-09-12T07:19:42.450230+0000","last_became_peered":"2024-09-12T07:19:42.450230+0000","last_unstale":"2024-09-12T07:20:17.425624+0000","last_undegraded":"2024-09-12T07:20:17.425624+0000","last_fullsized":"2024-09-12T07:20:17.425624+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:36.565217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076460899999999997,"stat_sum":{"num_bytes":29998209,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":4671,"num_read_kb":25061,"num_write":217,"num_write_kb":1817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"57'97","reported_seq":665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.307244+0000","last_change":"2024-09-12T07:19:44.480981+0000","last_active":"2024-09-12T07:20:17.307244+0000","last_peered":"2024-09-12T07:20:17.307244+0000","last_clean":"2024-09-12T07:20:17.307244+0000","last_became_active":"2024-09-12T07:19:42.452637+0000","last_became_peered":"2024-09-12T07:19:42.452637+0000","last_unstale":"2024-09-12T07:20:17.307244+0000","last_undegraded":"2024-09-12T07:20:17.307244+0000","last_fullsized":"2024-09-12T07:20:17.307244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:58:19.525415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000680892,"stat_sum":{"num_bytes":11712168,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":593,"num_read_kb":7565,"num_write":104,"num_write_kb":4494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"55'127","reported_seq":222,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.274107+0000","last_change":"2024-09-12T07:20:07.379907+0000","last_active":"2024-09-12T07:20:17.274107+0000","last_peered":"2024-09-12T07:20:17.274107+0000","last_clean":"2024-09-12T07:20:17.274107+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:20:17.274107+0000","last_undegraded":"2024-09-12T07:20:17.274107+0000","last_fullsized":"2024-09-12T07:20:17.274107+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:02.142020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054644100000000005,"stat_sum":{"num_bytes":11738737,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":1016,"num_read_kb":4367,"num_write":149,"num_write_kb":9733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"57'75","reported_seq":1633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.371568+0000","last_change":"2024-09-12T07:20:07.249347+0000","last_active":"2024-09-12T07:20:17.371568+0000","last_peered":"2024-09-12T07:20:17.371568+0000","last_clean":"2024-09-12T07:20:17.371568+0000","last_became_active":"2024-09-12T07:20:07.183065+0000","last_became_peered":"2024-09-12T07:20:07.183065+0000","last_unstale":"2024-09-12T07:20:17.371568+0000","last_undegraded":"2024-09-12T07:20:17.371568+0000","last_fullsized":"2024-09-12T07:20:17.371568+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:29.900552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050446300000000003,"stat_sum":{"num_bytes":20312597,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":1481,"num_read_kb":6145,"num_write":69,"num_write_kb":12726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":8578058,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"57'100","reported_seq":808,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.317589+0000","last_change":"2024-09-12T07:19:44.482470+0000","last_active":"2024-09-12T07:20:17.317589+0000","last_peered":"2024-09-12T07:20:17.317589+0000","last_clean":"2024-09-12T07:20:17.317589+0000","last_became_active":"2024-09-12T07:19:42.450577+0000","last_became_peered":"2024-09-12T07:19:42.450577+0000","last_unstale":"2024-09-12T07:20:17.317589+0000","last_undegraded":"2024-09-12T07:20:17.317589+0000","last_fullsized":"2024-09-12T07:20:17.317589+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:25:06.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317836,"stat_sum":{"num_bytes":17183410,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":968,"num_read_kb":9066,"num_write":114,"num_write_kb":2949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"57'274","reported_seq":1350,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.339612+0000","last_change":"2024-09-12T07:19:44.481850+0000","last_active":"2024-09-12T07:20:17.339612+0000","last_peered":"2024-09-12T07:20:17.339612+0000","last_clean":"2024-09-12T07:20:17.339612+0000","last_became_active":"2024-09-12T07:19:42.452236+0000","last_became_peered":"2024-09-12T07:19:42.452236+0000","last_unstale":"2024-09-12T07:20:17.339612+0000","last_undegraded":"2024-09-12T07:20:17.339612+0000","last_fullsized":"2024-09-12T07:20:17.339612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:53.534115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012939710000000001,"stat_sum":{"num_bytes":3004365,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":2844,"num_read_kb":3815,"num_write":371,"num_write_kb":1999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"57'259","reported_seq":804,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.242263+0000","last_change":"2024-09-12T07:19:44.482179+0000","last_active":"2024-09-12T07:20:17.242263+0000","last_peered":"2024-09-12T07:20:17.242263+0000","last_clean":"2024-09-12T07:20:17.242263+0000","last_became_active":"2024-09-12T07:19:42.452155+0000","last_became_peered":"2024-09-12T07:19:42.452155+0000","last_unstale":"2024-09-12T07:20:17.242263+0000","last_undegraded":"2024-09-12T07:20:17.242263+0000","last_fullsized":"2024-09-12T07:20:17.242263+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.797732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011884759999999999,"stat_sum":{"num_bytes":13654463,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":1668,"num_read_kb":5871,"num_write":326,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"57'60","reported_seq":236,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.289322+0000","last_change":"2024-09-12T07:19:44.481020+0000","last_active":"2024-09-12T07:20:17.289322+0000","last_peered":"2024-09-12T07:20:17.289322+0000","last_clean":"2024-09-12T07:20:17.289322+0000","last_became_active":"2024-09-12T07:19:42.450535+0000","last_became_peered":"2024-09-12T07:19:42.450535+0000","last_unstale":"2024-09-12T07:20:17.289322+0000","last_undegraded":"2024-09-12T07:20:17.289322+0000","last_fullsized":"2024-09-12T07:20:17.289322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:49.955754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073032100000000003,"stat_sum":{"num_bytes":5271552,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":107,"num_read_kb":2266,"num_write":59,"num_write_kb":5105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"55'71","reported_seq":2402,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.242160+0000","last_change":"2024-09-12T07:19:44.481275+0000","last_active":"2024-09-12T07:20:17.242160+0000","last_peered":"2024-09-12T07:20:17.242160+0000","last_clean":"2024-09-12T07:20:17.242160+0000","last_became_active":"2024-09-12T07:19:42.452104+0000","last_became_peered":"2024-09-12T07:19:42.452104+0000","last_unstale":"2024-09-12T07:20:17.242160+0000","last_undegraded":"2024-09-12T07:20:17.242160+0000","last_fullsized":"2024-09-12T07:20:17.242160+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:26:53.102880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079083699999999996,"stat_sum":{"num_bytes":15311876,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":2262,"num_read_kb":29393,"num_write":71,"num_write_kb":11641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"54'57","reported_seq":1428,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.232408+0000","last_change":"2024-09-12T07:19:44.456086+0000","last_active":"2024-09-12T07:20:17.232408+0000","last_peered":"2024-09-12T07:20:17.232408+0000","last_clean":"2024-09-12T07:20:17.232408+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:20:17.232408+0000","last_undegraded":"2024-09-12T07:20:17.232408+0000","last_fullsized":"2024-09-12T07:20:17.232408+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:48:15.796361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060746,"stat_sum":{"num_bytes":9766249,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":1285,"num_read_kb":17355,"num_write":59,"num_write_kb":2111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"55'225","reported_seq":16398,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.242197+0000","last_change":"2024-09-12T07:19:44.482564+0000","last_active":"2024-09-12T07:20:17.242197+0000","last_peered":"2024-09-12T07:20:17.242197+0000","last_clean":"2024-09-12T07:20:17.242197+0000","last_became_active":"2024-09-12T07:19:42.456127+0000","last_became_peered":"2024-09-12T07:19:42.456127+0000","last_unstale":"2024-09-12T07:20:17.242197+0000","last_undegraded":"2024-09-12T07:20:17.242197+0000","last_fullsized":"2024-09-12T07:20:17.242197+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:25:18.311652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793039999999999,"stat_sum":{"num_bytes":36095087,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":16684,"num_read_kb":47561,"num_write":297,"num_write_kb":24067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":560283846,"num_objects":650,"num_object_clones":134,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":650,"num_whiteouts":268,"num_read":111144,"num_read_kb":411035,"num_write":5763,"num_write_kb":282020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":49942512,"num_keys_recovered":48,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434601984,"data_stored":462173036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4721,"ondisk_log_size":4721,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100964,"kb_used_data":69772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94270876,"statfs":{"total":96636764160,"available":96533377024,"internally_reserved":0,"allocated":71446528,"data_stored":73019625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50488,"kb_used_data":19296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321352,"statfs":{"total":96636764160,"available":96585064448,"internally_reserved":0,"allocated":19759104,"data_stored":19378068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100052,"kb_used_data":64708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94271788,"statfs":{"total":96636764160,"available":96534310912,"internally_reserved":0,"allocated":66260992,"data_stored":70451527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542157,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":142824,"kb_used_data":107476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94229016,"statfs":{"total":96636764160,"available":96490512384,"internally_reserved":0,"allocated":110055424,"data_stored":116991307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108832,"kb_used_data":69304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94263008,"statfs":{"total":96636764160,"available":96525320192,"internally_reserved":0,"allocated":70967296,"data_stored":70427411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542158,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44892,"kb_used_data":9536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326948,"statfs":{"total":96636764160,"available":96590794752,"internally_reserved":0,"allocated":9764864,"data_stored":9371476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65360,"kb_used_data":29968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94306480,"statfs":{"total":96636764160,"available":96569835520,"internally_reserved":0,"allocated":30687232,"data_stored":30932502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95940,"kb_used_data":60564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94275900,"statfs":{"total":96636764160,"available":96538521600,"internally_reserved":0,"allocated":62017536,"data_stored":64798491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60780544,"data_stored":64860235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30044160,"data_stored":31584611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9109504,"data_stored":9083549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":70332416,"data_stored":72938934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":109400064,"data_stored":117637258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65613824,"data_stored":73896863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":19103744,"data_stored":19090141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":70217728,"data_stored":73081445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:20:19.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:20:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:2419 trunc from 0x672cbf to 0x2ad260 2024-09-12T07:20:19.718 INFO:teuthology.orchestra.run.smithi046.stdout:2421 writesame 0xbb01d92 thru 0xbb08234 data_size 0x1(0x64a3 bytes) 2024-09-12T07:20:19.718 INFO:teuthology.orchestra.run.smithi046.stdout:2423 read 0x41537d2 thru 0x41551f8 (0x1a27 bytes) 2024-09-12T07:20:19.718 INFO:teuthology.orchestra.run.smithi046.stdout:2424 punch from 0x9ef7b74 to 0x9f057ae, (0xdc3a bytes) 2024-09-12T07:20:19.722 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:19.723+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:19.730 INFO:teuthology.orchestra.run.smithi046.stdout:2426 trunc from 0xbb08235 to 0xf92180 2024-09-12T07:20:19.735 INFO:teuthology.orchestra.run.smithi046.stdout:2427 punch from 0xcafec7 to 0xcba692, (0xa7cb bytes) 2024-09-12T07:20:19.745 INFO:teuthology.orchestra.run.smithi046.stdout:2428 write 0x5505bc1 thru 0x550f9c0 (0x9e00 bytes) 2024-09-12T07:20:19.748 INFO:teuthology.orchestra.run.smithi046.stdout:2429 write 0x25ed5bc thru 0x25ee70d (0x1152 bytes) 2024-09-12T07:20:19.760 INFO:teuthology.orchestra.run.smithi046.stdout:2430 write 0x856bf74 thru 0x8571935 (0x59c2 bytes) 2024-09-12T07:20:19.763 INFO:teuthology.orchestra.run.smithi046.stdout:2433 punch from 0x3c006b0 to 0x3c0f332, (0xec82 bytes) 2024-09-12T07:20:19.766 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:19.766 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:19.768 INFO:teuthology.orchestra.run.smithi046.stdout:2434 writesame 0x896429b thru 0x896c1fa data_size 0x10(0x7f60 bytes) 2024-09-12T07:20:19.774 INFO:teuthology.orchestra.run.smithi046.stdout:2435 writesame 0x95cf299 thru 0x95d0802 data_size 0x1(0x156a bytes) 2024-09-12T07:20:19.774 INFO:teuthology.orchestra.run.smithi046.stdout:2436 writesame 0x4459dc0 thru 0x445a133 data_size 0xd(0x374 bytes) 2024-09-12T07:20:19.775 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:19.783 INFO:teuthology.orchestra.run.smithi046.stdout:2438 write 0xa4b838b thru 0xa4c3456 (0xb0cc bytes) 2024-09-12T07:20:19.787 INFO:teuthology.orchestra.run.smithi046.stdout:2440 read 0x3b2552a thru 0x3b332c7 (0xdd9e bytes) 2024-09-12T07:20:19.787 INFO:teuthology.orchestra.run.smithi046.stdout:2441 punch from 0x55224c7 to 0x55239c7, (0x1500 bytes) 2024-09-12T07:20:19.794 INFO:teuthology.orchestra.run.smithi046.stdout:2443 writesame 0xc8ded8e thru 0xc8e23d8 data_size 0x3(0x364b bytes) 2024-09-12T07:20:19.796 INFO:teuthology.orchestra.run.smithi046.stdout:2445 trunc from 0xc8e23d9 to 0xba1e0b8 2024-09-12T07:20:19.801 INFO:teuthology.orchestra.run.smithi046.stdout:2446 writesame 0x95368d6 thru 0x9537423 data_size 0x1(0xb4e bytes) 2024-09-12T07:20:19.805 INFO:teuthology.orchestra.run.smithi046.stdout:2448 write 0x15d4452 thru 0x15dd458 (0x9007 bytes) 2024-09-12T07:20:19.819 INFO:teuthology.orchestra.run.smithi046.stdout:2450 trunc from 0xba1e0b8 to 0x27322c5 2024-09-12T07:20:19.830 INFO:teuthology.orchestra.run.smithi046.stdout:2451 punch from 0x208914e to 0x2098f88, (0xfe3a bytes) 2024-09-12T07:20:19.830 INFO:teuthology.orchestra.run.smithi046.stdout:2453 read 0x372721 thru 0x3783e1 (0x5cc1 bytes) 2024-09-12T07:20:19.841 INFO:teuthology.orchestra.run.smithi046.stdout:2454 write 0x7b75b60 thru 0x7b777b7 (0x1c58 bytes) 2024-09-12T07:20:19.844 INFO:teuthology.orchestra.run.smithi046.stdout:2455 read 0x21b17c9 thru 0x21bcf93 (0xb7cb bytes) 2024-09-12T07:20:19.844 INFO:teuthology.orchestra.run.smithi046.stdout:2456 punch from 0x1fc5d93 to 0x1fd080d, (0xaa7a bytes) 2024-09-12T07:20:19.857 INFO:teuthology.orchestra.run.smithi046.stdout:2458 write 0xe2708e0 thru 0xe27fdcc (0xf4ed bytes) 2024-09-12T07:20:19.859 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:19.860+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:19.860 INFO:teuthology.orchestra.run.smithi046.stdout:2460 write 0x826d945 thru 0x827ba9e (0xe15a bytes) 2024-09-12T07:20:19.863 INFO:teuthology.orchestra.run.smithi046.stdout:2461 write 0x1261c59 thru 0x126735c (0x5704 bytes) 2024-09-12T07:20:19.867 INFO:teuthology.orchestra.run.smithi046.stdout:2462 writesame 0x72f0f47 thru 0x72fe66e data_size 0x99(0xd728 bytes) 2024-09-12T07:20:19.870 INFO:teuthology.orchestra.run.smithi046.stdout:2464 writesame 0x494607b thru 0x4953833 data_size 0x2f(0xd7b9 bytes) 2024-09-12T07:20:19.873 INFO:teuthology.orchestra.run.smithi046.stdout:2465 write 0x486020b thru 0x48637f0 (0x35e6 bytes) 2024-09-12T07:20:19.876 INFO:teuthology.orchestra.run.smithi046.stdout:2466 read 0x7c55844 thru 0x7c5975d (0x3f1a bytes) 2024-09-12T07:20:19.876 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:19.885 INFO:teuthology.orchestra.run.smithi046.stdout:2468 trunc from 0xe27fdcd to 0x6ef5800 2024-09-12T07:20:19.900 INFO:teuthology.orchestra.run.smithi046.stdout:2469 write 0x76116d8 thru 0x7611b7d (0x4a6 bytes) 2024-09-12T07:20:19.903 INFO:teuthology.orchestra.run.smithi046.stdout:2470 clone 18 order 24 su 8388608 sc 13 2024-09-12T07:20:19.924 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:19.924 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:19.925 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:19.926+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:19.933 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:20.019 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:20:20.020 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:20:20.025 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:20.026+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:20.032 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-12T07:20:19.499201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":560874233,"num_objects":654,"num_object_clones":134,"num_object_copies":1308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":654,"num_whiteouts":268,"num_read":111210,"num_read_kb":411090,"num_write":5866,"num_write_kb":283650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":49942512,"num_keys_recovered":48,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4796,"ondisk_log_size":4796,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":709352,"kb_used_data":430624,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754265368,"statfs":{"total":773094113280,"available":772367736832,"internally_reserved":0,"allocated":440958976,"data_stored":455370407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":150503677,"num_objects":246,"num_object_clones":47,"num_object_copies":492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":246,"num_whiteouts":153,"num_read":11529,"num_read_kb":92519,"num_write":1798,"num_write_kb":92725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000569"},"pg_stats":[{"pgid":"3.1f","version":"57'242","reported_seq":1844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.232649+0000","last_change":"2024-09-12T07:19:44.455827+0000","last_active":"2024-09-12T07:20:17.232649+0000","last_peered":"2024-09-12T07:20:17.232649+0000","last_clean":"2024-09-12T07:20:17.232649+0000","last_became_active":"2024-09-12T07:19:42.451212+0000","last_became_peered":"2024-09-12T07:19:42.451212+0000","last_unstale":"2024-09-12T07:20:17.232649+0000","last_undegraded":"2024-09-12T07:20:17.232649+0000","last_fullsized":"2024-09-12T07:20:17.232649+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:57.585258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001023672,"stat_sum":{"num_bytes":10940340,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":2844,"num_read_kb":15275,"num_write":311,"num_write_kb":4391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"57'118","reported_seq":141,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.274011+0000","last_change":"2024-09-12T07:20:07.305432+0000","last_active":"2024-09-12T07:20:17.274011+0000","last_peered":"2024-09-12T07:20:17.274011+0000","last_clean":"2024-09-12T07:20:17.274011+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:20:17.274011+0000","last_undegraded":"2024-09-12T07:20:17.274011+0000","last_fullsized":"2024-09-12T07:20:17.274011+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:58.448307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081935099999999996,"stat_sum":{"num_bytes":28822701,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":16696,"num_read_kb":20572,"num_write":107,"num_write_kb":16946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"57'98","reported_seq":17107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.252898+0000","last_change":"2024-09-12T07:19:44.464376+0000","last_active":"2024-09-12T07:20:17.252898+0000","last_peered":"2024-09-12T07:20:17.252898+0000","last_clean":"2024-09-12T07:20:17.252898+0000","last_became_active":"2024-09-12T07:19:42.451973+0000","last_became_peered":"2024-09-12T07:19:42.451973+0000","last_unstale":"2024-09-12T07:20:17.252898+0000","last_undegraded":"2024-09-12T07:20:17.252898+0000","last_fullsized":"2024-09-12T07:20:17.252898+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:27.976836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036949999999999,"stat_sum":{"num_bytes":26782347,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":16959,"num_read_kb":23181,"num_write":89,"num_write_kb":10322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'144","reported_seq":1698,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.358238+0000","last_change":"2024-09-12T07:19:44.464287+0000","last_active":"2024-09-12T07:20:17.358238+0000","last_peered":"2024-09-12T07:20:17.358238+0000","last_clean":"2024-09-12T07:20:17.358238+0000","last_became_active":"2024-09-12T07:19:42.455338+0000","last_became_peered":"2024-09-12T07:19:42.455338+0000","last_unstale":"2024-09-12T07:20:17.358238+0000","last_undegraded":"2024-09-12T07:20:17.358238+0000","last_fullsized":"2024-09-12T07:20:17.358238+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:50.776177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012347180000000001,"stat_sum":{"num_bytes":13823415,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":1857,"num_read_kb":9757,"num_write":182,"num_write_kb":2570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"56'97","reported_seq":12959,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.232468+0000","last_change":"2024-09-12T07:19:44.456652+0000","last_active":"2024-09-12T07:20:17.232468+0000","last_peered":"2024-09-12T07:20:17.232468+0000","last_clean":"2024-09-12T07:20:17.232468+0000","last_became_active":"2024-09-12T07:19:42.454990+0000","last_became_peered":"2024-09-12T07:19:42.454990+0000","last_unstale":"2024-09-12T07:20:17.232468+0000","last_undegraded":"2024-09-12T07:20:17.232468+0000","last_fullsized":"2024-09-12T07:20:17.232468+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:13:53.218197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049866,"stat_sum":{"num_bytes":20900457,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":13242,"num_read_kb":14669,"num_write":88,"num_write_kb":11790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"57'114","reported_seq":1053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.377750+0000","last_change":"2024-09-12T07:19:44.459417+0000","last_active":"2024-09-12T07:20:17.377750+0000","last_peered":"2024-09-12T07:20:17.377750+0000","last_clean":"2024-09-12T07:20:17.377750+0000","last_became_active":"2024-09-12T07:19:42.454620+0000","last_became_peered":"2024-09-12T07:19:42.454620+0000","last_unstale":"2024-09-12T07:20:17.377750+0000","last_undegraded":"2024-09-12T07:20:17.377750+0000","last_fullsized":"2024-09-12T07:20:17.377750+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:06.854794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090501500000000001,"stat_sum":{"num_bytes":13596786,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":880,"num_read_kb":2263,"num_write":108,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"57'243","reported_seq":1072,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.422045+0000","last_change":"2024-09-12T07:20:10.196744+0000","last_active":"2024-09-12T07:20:17.422045+0000","last_peered":"2024-09-12T07:20:17.422045+0000","last_clean":"2024-09-12T07:20:17.422045+0000","last_became_active":"2024-09-12T07:20:10.195312+0000","last_became_peered":"2024-09-12T07:20:10.195312+0000","last_unstale":"2024-09-12T07:20:17.422045+0000","last_undegraded":"2024-09-12T07:20:17.422045+0000","last_fullsized":"2024-09-12T07:20:17.422045+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:13.894288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113156,"stat_sum":{"num_bytes":16795101,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":1503,"num_read_kb":10040,"num_write":354,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1101525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"54'118","reported_seq":282,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.274002+0000","last_change":"2024-09-12T07:20:07.446518+0000","last_active":"2024-09-12T07:20:17.274002+0000","last_peered":"2024-09-12T07:20:17.274002+0000","last_clean":"2024-09-12T07:20:17.274002+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:20:17.274002+0000","last_undegraded":"2024-09-12T07:20:17.274002+0000","last_fullsized":"2024-09-12T07:20:17.274002+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:36:33.763137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000443294,"stat_sum":{"num_bytes":22996397,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":2082,"num_read_kb":16378,"num_write":123,"num_write_kb":5396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"55'60","reported_seq":2447,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.232849+0000","last_change":"2024-09-12T07:19:44.456115+0000","last_active":"2024-09-12T07:20:17.232849+0000","last_peered":"2024-09-12T07:20:17.232849+0000","last_clean":"2024-09-12T07:20:17.232849+0000","last_became_active":"2024-09-12T07:19:42.452617+0000","last_became_peered":"2024-09-12T07:19:42.452617+0000","last_unstale":"2024-09-12T07:20:17.232849+0000","last_undegraded":"2024-09-12T07:20:17.232849+0000","last_fullsized":"2024-09-12T07:20:17.232849+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:25.151293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077251,"stat_sum":{"num_bytes":24337497,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":2312,"num_read_kb":9400,"num_write":63,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"57'59","reported_seq":160,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.317817+0000","last_change":"2024-09-12T07:19:44.463507+0000","last_active":"2024-09-12T07:20:17.317817+0000","last_peered":"2024-09-12T07:20:17.317817+0000","last_clean":"2024-09-12T07:20:17.317817+0000","last_became_active":"2024-09-12T07:19:42.455210+0000","last_became_peered":"2024-09-12T07:19:42.455210+0000","last_unstale":"2024-09-12T07:20:17.317817+0000","last_undegraded":"2024-09-12T07:20:17.317817+0000","last_fullsized":"2024-09-12T07:20:17.317817+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:12.237223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049753700000000004,"stat_sum":{"num_bytes":12309059,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":8,"num_read":25,"num_read_kb":108,"num_write":51,"num_write_kb":4628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"57'86","reported_seq":410,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.309870+0000","last_change":"2024-09-12T07:20:07.261139+0000","last_active":"2024-09-12T07:20:17.309870+0000","last_peered":"2024-09-12T07:20:17.309870+0000","last_clean":"2024-09-12T07:20:17.309870+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:20:17.309870+0000","last_undegraded":"2024-09-12T07:20:17.309870+0000","last_fullsized":"2024-09-12T07:20:17.309870+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:57:02.011265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012225490000000001,"stat_sum":{"num_bytes":7216656,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":487,"num_read_kb":1445,"num_write":72,"num_write_kb":4344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"57'121","reported_seq":3850,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.370365+0000","last_change":"2024-09-12T07:19:44.481796+0000","last_active":"2024-09-12T07:20:17.370365+0000","last_peered":"2024-09-12T07:20:17.370365+0000","last_clean":"2024-09-12T07:20:17.370365+0000","last_became_active":"2024-09-12T07:19:42.452277+0000","last_became_peered":"2024-09-12T07:19:42.452277+0000","last_unstale":"2024-09-12T07:20:17.370365+0000","last_undegraded":"2024-09-12T07:20:17.370365+0000","last_fullsized":"2024-09-12T07:20:17.370365+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:59.618843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012485160000000001,"stat_sum":{"num_bytes":31451428,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":4114,"num_read_kb":49211,"num_write":128,"num_write_kb":23849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.289203+0000","last_change":"2024-09-12T07:19:32.474512+0000","last_active":"2024-09-12T07:20:17.289203+0000","last_peered":"2024-09-12T07:20:17.289203+0000","last_clean":"2024-09-12T07:20:17.289203+0000","last_became_active":"2024-09-12T07:19:30.219990+0000","last_became_peered":"2024-09-12T07:19:30.219990+0000","last_unstale":"2024-09-12T07:20:17.289203+0000","last_undegraded":"2024-09-12T07:20:17.289203+0000","last_fullsized":"2024-09-12T07:20:17.289203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:32.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046339800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"57'261","reported_seq":1203,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.421897+0000","last_change":"2024-09-12T07:19:44.455257+0000","last_active":"2024-09-12T07:20:17.421897+0000","last_peered":"2024-09-12T07:20:17.421897+0000","last_clean":"2024-09-12T07:20:17.421897+0000","last_became_active":"2024-09-12T07:19:42.449730+0000","last_became_peered":"2024-09-12T07:19:42.449730+0000","last_unstale":"2024-09-12T07:20:17.421897+0000","last_undegraded":"2024-09-12T07:20:17.421897+0000","last_fullsized":"2024-09-12T07:20:17.421897+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:18:04.089947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059259600000000003,"stat_sum":{"num_bytes":9633736,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":1792,"num_read_kb":8645,"num_write":308,"num_write_kb":1367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.235223+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:20:17.235223+0000","last_peered":"2024-09-12T07:20:17.235223+0000","last_clean":"2024-09-12T07:20:17.235223+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:20:17.235223+0000","last_undegraded":"2024-09-12T07:20:17.235223+0000","last_fullsized":"2024-09-12T07:20:17.235223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"57'76","reported_seq":685,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.422556+0000","last_change":"2024-09-12T07:20:07.211023+0000","last_active":"2024-09-12T07:20:17.422556+0000","last_peered":"2024-09-12T07:20:17.422556+0000","last_clean":"2024-09-12T07:20:17.422556+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:20:17.422556+0000","last_undegraded":"2024-09-12T07:20:17.422556+0000","last_fullsized":"2024-09-12T07:20:17.422556+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:45.978689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087343100000000003,"stat_sum":{"num_bytes":22331570,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":668,"num_read_kb":11201,"num_write":79,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.242263+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:20:17.242263+0000","last_peered":"2024-09-12T07:20:17.242263+0000","last_clean":"2024-09-12T07:20:17.242263+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:20:17.242263+0000","last_undegraded":"2024-09-12T07:20:17.242263+0000","last_fullsized":"2024-09-12T07:20:17.242263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"55'166","reported_seq":525,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.235066+0000","last_change":"2024-09-12T07:19:44.458925+0000","last_active":"2024-09-12T07:20:17.235066+0000","last_peered":"2024-09-12T07:20:17.235066+0000","last_clean":"2024-09-12T07:20:17.235066+0000","last_became_active":"2024-09-12T07:19:42.452704+0000","last_became_peered":"2024-09-12T07:19:42.452704+0000","last_unstale":"2024-09-12T07:20:17.235066+0000","last_undegraded":"2024-09-12T07:20:17.235066+0000","last_fullsized":"2024-09-12T07:20:17.235066+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:35.846662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045577200000000001,"stat_sum":{"num_bytes":2876510,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":1189,"num_read_kb":1980,"num_write":236,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.234965+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:20:17.234965+0000","last_peered":"2024-09-12T07:20:17.234965+0000","last_clean":"2024-09-12T07:20:17.234965+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:20:17.234965+0000","last_undegraded":"2024-09-12T07:20:17.234965+0000","last_fullsized":"2024-09-12T07:20:17.234965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"57'110","reported_seq":3254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.336522+0000","last_change":"2024-09-12T07:20:07.289858+0000","last_active":"2024-09-12T07:20:17.336522+0000","last_peered":"2024-09-12T07:20:17.336522+0000","last_clean":"2024-09-12T07:20:17.336522+0000","last_became_active":"2024-09-12T07:20:07.183819+0000","last_became_peered":"2024-09-12T07:20:07.183819+0000","last_unstale":"2024-09-12T07:20:17.336522+0000","last_undegraded":"2024-09-12T07:20:17.336522+0000","last_fullsized":"2024-09-12T07:20:17.336522+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:25:17.468030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015134619999999999,"stat_sum":{"num_bytes":44012764,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":3059,"num_read_kb":23450,"num_write":139,"num_write_kb":17370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":3070354,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"57'63","reported_seq":580,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.307609+0000","last_change":"2024-09-12T07:19:44.456465+0000","last_active":"2024-09-12T07:20:17.307609+0000","last_peered":"2024-09-12T07:20:17.307609+0000","last_clean":"2024-09-12T07:20:17.307609+0000","last_became_active":"2024-09-12T07:19:42.454103+0000","last_became_peered":"2024-09-12T07:19:42.454103+0000","last_unstale":"2024-09-12T07:20:17.307609+0000","last_undegraded":"2024-09-12T07:20:17.307609+0000","last_fullsized":"2024-09-12T07:20:17.307609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:11.047251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010980880000000001,"stat_sum":{"num_bytes":7410324,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":14,"num_read":441,"num_read_kb":1640,"num_write":47,"num_write_kb":1264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.252568+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:20:17.252568+0000","last_peered":"2024-09-12T07:20:17.252568+0000","last_clean":"2024-09-12T07:20:17.252568+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:20:17.252568+0000","last_undegraded":"2024-09-12T07:20:17.252568+0000","last_fullsized":"2024-09-12T07:20:17.252568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"53'135","reported_seq":818,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.235066+0000","last_change":"2024-09-12T07:19:44.458868+0000","last_active":"2024-09-12T07:20:17.235066+0000","last_peered":"2024-09-12T07:20:17.235066+0000","last_clean":"2024-09-12T07:20:17.235066+0000","last_became_active":"2024-09-12T07:19:42.451807+0000","last_became_peered":"2024-09-12T07:19:42.451807+0000","last_unstale":"2024-09-12T07:20:17.235066+0000","last_undegraded":"2024-09-12T07:20:17.235066+0000","last_fullsized":"2024-09-12T07:20:17.235066+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.743040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048363399999999998,"stat_sum":{"num_bytes":12607390,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":1548,"num_read_kb":3970,"num_write":144,"num_write_kb":8640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.229409+0000","last_change":"2024-09-12T07:20:07.183543+0000","last_active":"2024-09-12T07:20:17.229409+0000","last_peered":"2024-09-12T07:20:17.229409+0000","last_clean":"2024-09-12T07:20:17.229409+0000","last_became_active":"2024-09-12T07:20:07.183213+0000","last_became_peered":"2024-09-12T07:20:07.183213+0000","last_unstale":"2024-09-12T07:20:17.229409+0000","last_undegraded":"2024-09-12T07:20:17.229409+0000","last_fullsized":"2024-09-12T07:20:17.229409+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.634459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"57'93","reported_seq":1073,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.328840+0000","last_change":"2024-09-12T07:19:44.455560+0000","last_active":"2024-09-12T07:20:17.328840+0000","last_peered":"2024-09-12T07:20:17.328840+0000","last_clean":"2024-09-12T07:20:17.328840+0000","last_became_active":"2024-09-12T07:19:42.451970+0000","last_became_peered":"2024-09-12T07:19:42.451970+0000","last_unstale":"2024-09-12T07:20:17.328840+0000","last_undegraded":"2024-09-12T07:20:17.328840+0000","last_fullsized":"2024-09-12T07:20:17.328840+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:26.326667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083352100000000004,"stat_sum":{"num_bytes":18618824,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":1836,"num_read_kb":10738,"num_write":98,"num_write_kb":15485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.241930+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:20:17.241930+0000","last_peered":"2024-09-12T07:20:17.241930+0000","last_clean":"2024-09-12T07:20:17.241930+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:20:17.241930+0000","last_undegraded":"2024-09-12T07:20:17.241930+0000","last_fullsized":"2024-09-12T07:20:17.241930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"57'623","reported_seq":1113,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.374159+0000","last_change":"2024-09-12T07:20:07.467030+0000","last_active":"2024-09-12T07:20:17.374159+0000","last_peered":"2024-09-12T07:20:17.374159+0000","last_clean":"2024-09-12T07:20:17.374159+0000","last_became_active":"2024-09-12T07:20:07.180577+0000","last_became_peered":"2024-09-12T07:20:07.180577+0000","last_unstale":"2024-09-12T07:20:17.374159+0000","last_undegraded":"2024-09-12T07:20:17.374159+0000","last_fullsized":"2024-09-12T07:20:17.374159+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:47.344526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089638599999999999,"stat_sum":{"num_bytes":15519005,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":3427,"num_read_kb":9930,"num_write":1046,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":261251,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":91,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.252567+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:20:17.252567+0000","last_peered":"2024-09-12T07:20:17.252567+0000","last_clean":"2024-09-12T07:20:17.252567+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:20:17.252567+0000","last_undegraded":"2024-09-12T07:20:17.252567+0000","last_fullsized":"2024-09-12T07:20:17.252567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":181,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.241930+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:20:17.241930+0000","last_peered":"2024-09-12T07:20:17.241930+0000","last_clean":"2024-09-12T07:20:17.241930+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:20:17.241930+0000","last_undegraded":"2024-09-12T07:20:17.241930+0000","last_fullsized":"2024-09-12T07:20:17.241930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"57'114","reported_seq":503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.310049+0000","last_change":"2024-09-12T07:20:07.428482+0000","last_active":"2024-09-12T07:20:17.310049+0000","last_peered":"2024-09-12T07:20:17.310049+0000","last_clean":"2024-09-12T07:20:17.310049+0000","last_became_active":"2024-09-12T07:20:07.180763+0000","last_became_peered":"2024-09-12T07:20:07.180763+0000","last_unstale":"2024-09-12T07:20:17.310049+0000","last_undegraded":"2024-09-12T07:20:17.310049+0000","last_fullsized":"2024-09-12T07:20:17.310049+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:33.693067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047491800000000001,"stat_sum":{"num_bytes":23252826,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":7,"num_read":604,"num_read_kb":8717,"num_write":154,"num_write_kb":14586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":8633990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"57'235","reported_seq":3231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.425624+0000","last_change":"2024-09-12T07:19:44.469508+0000","last_active":"2024-09-12T07:20:17.425624+0000","last_peered":"2024-09-12T07:20:17.425624+0000","last_clean":"2024-09-12T07:20:17.425624+0000","last_became_active":"2024-09-12T07:19:42.450230+0000","last_became_peered":"2024-09-12T07:19:42.450230+0000","last_unstale":"2024-09-12T07:20:17.425624+0000","last_undegraded":"2024-09-12T07:20:17.425624+0000","last_fullsized":"2024-09-12T07:20:17.425624+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:36.565217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076460899999999997,"stat_sum":{"num_bytes":29998209,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":4671,"num_read_kb":25061,"num_write":217,"num_write_kb":1817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"57'97","reported_seq":665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.307244+0000","last_change":"2024-09-12T07:19:44.480981+0000","last_active":"2024-09-12T07:20:17.307244+0000","last_peered":"2024-09-12T07:20:17.307244+0000","last_clean":"2024-09-12T07:20:17.307244+0000","last_became_active":"2024-09-12T07:19:42.452637+0000","last_became_peered":"2024-09-12T07:19:42.452637+0000","last_unstale":"2024-09-12T07:20:17.307244+0000","last_undegraded":"2024-09-12T07:20:17.307244+0000","last_fullsized":"2024-09-12T07:20:17.307244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:58:19.525415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000680892,"stat_sum":{"num_bytes":11712168,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":593,"num_read_kb":7565,"num_write":104,"num_write_kb":4494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"55'127","reported_seq":222,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.274107+0000","last_change":"2024-09-12T07:20:07.379907+0000","last_active":"2024-09-12T07:20:17.274107+0000","last_peered":"2024-09-12T07:20:17.274107+0000","last_clean":"2024-09-12T07:20:17.274107+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:20:17.274107+0000","last_undegraded":"2024-09-12T07:20:17.274107+0000","last_fullsized":"2024-09-12T07:20:17.274107+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:02.142020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054644100000000005,"stat_sum":{"num_bytes":11738737,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":1016,"num_read_kb":4367,"num_write":149,"num_write_kb":9733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"57'75","reported_seq":1633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.371568+0000","last_change":"2024-09-12T07:20:07.249347+0000","last_active":"2024-09-12T07:20:17.371568+0000","last_peered":"2024-09-12T07:20:17.371568+0000","last_clean":"2024-09-12T07:20:17.371568+0000","last_became_active":"2024-09-12T07:20:07.183065+0000","last_became_peered":"2024-09-12T07:20:07.183065+0000","last_unstale":"2024-09-12T07:20:17.371568+0000","last_undegraded":"2024-09-12T07:20:17.371568+0000","last_fullsized":"2024-09-12T07:20:17.371568+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:29.900552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050446300000000003,"stat_sum":{"num_bytes":20312597,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":1481,"num_read_kb":6145,"num_write":69,"num_write_kb":12726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":8578058,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"57'100","reported_seq":808,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.317589+0000","last_change":"2024-09-12T07:19:44.482470+0000","last_active":"2024-09-12T07:20:17.317589+0000","last_peered":"2024-09-12T07:20:17.317589+0000","last_clean":"2024-09-12T07:20:17.317589+0000","last_became_active":"2024-09-12T07:19:42.450577+0000","last_became_peered":"2024-09-12T07:19:42.450577+0000","last_unstale":"2024-09-12T07:20:17.317589+0000","last_undegraded":"2024-09-12T07:20:17.317589+0000","last_fullsized":"2024-09-12T07:20:17.317589+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:25:06.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317836,"stat_sum":{"num_bytes":17183410,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":968,"num_read_kb":9066,"num_write":114,"num_write_kb":2949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"57'274","reported_seq":1350,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.339612+0000","last_change":"2024-09-12T07:19:44.481850+0000","last_active":"2024-09-12T07:20:17.339612+0000","last_peered":"2024-09-12T07:20:17.339612+0000","last_clean":"2024-09-12T07:20:17.339612+0000","last_became_active":"2024-09-12T07:19:42.452236+0000","last_became_peered":"2024-09-12T07:19:42.452236+0000","last_unstale":"2024-09-12T07:20:17.339612+0000","last_undegraded":"2024-09-12T07:20:17.339612+0000","last_fullsized":"2024-09-12T07:20:17.339612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:53.534115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012939710000000001,"stat_sum":{"num_bytes":3004365,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":2844,"num_read_kb":3815,"num_write":371,"num_write_kb":1999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"57'259","reported_seq":804,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.242263+0000","last_change":"2024-09-12T07:19:44.482179+0000","last_active":"2024-09-12T07:20:17.242263+0000","last_peered":"2024-09-12T07:20:17.242263+0000","last_clean":"2024-09-12T07:20:17.242263+0000","last_became_active":"2024-09-12T07:19:42.452155+0000","last_became_peered":"2024-09-12T07:19:42.452155+0000","last_unstale":"2024-09-12T07:20:17.242263+0000","last_undegraded":"2024-09-12T07:20:17.242263+0000","last_fullsized":"2024-09-12T07:20:17.242263+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.797732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011884759999999999,"stat_sum":{"num_bytes":13654463,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":1668,"num_read_kb":5871,"num_write":326,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"57'60","reported_seq":236,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.289322+0000","last_change":"2024-09-12T07:19:44.481020+0000","last_active":"2024-09-12T07:20:17.289322+0000","last_peered":"2024-09-12T07:20:17.289322+0000","last_clean":"2024-09-12T07:20:17.289322+0000","last_became_active":"2024-09-12T07:19:42.450535+0000","last_became_peered":"2024-09-12T07:19:42.450535+0000","last_unstale":"2024-09-12T07:20:17.289322+0000","last_undegraded":"2024-09-12T07:20:17.289322+0000","last_fullsized":"2024-09-12T07:20:17.289322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:49.955754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073032100000000003,"stat_sum":{"num_bytes":5271552,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":107,"num_read_kb":2266,"num_write":59,"num_write_kb":5105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"55'71","reported_seq":2402,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.242160+0000","last_change":"2024-09-12T07:19:44.481275+0000","last_active":"2024-09-12T07:20:17.242160+0000","last_peered":"2024-09-12T07:20:17.242160+0000","last_clean":"2024-09-12T07:20:17.242160+0000","last_became_active":"2024-09-12T07:19:42.452104+0000","last_became_peered":"2024-09-12T07:19:42.452104+0000","last_unstale":"2024-09-12T07:20:17.242160+0000","last_undegraded":"2024-09-12T07:20:17.242160+0000","last_fullsized":"2024-09-12T07:20:17.242160+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:26:53.102880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079083699999999996,"stat_sum":{"num_bytes":15311876,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":2262,"num_read_kb":29393,"num_write":71,"num_write_kb":11641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"54'57","reported_seq":1428,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.232408+0000","last_change":"2024-09-12T07:19:44.456086+0000","last_active":"2024-09-12T07:20:17.232408+0000","last_peered":"2024-09-12T07:20:17.232408+0000","last_clean":"2024-09-12T07:20:17.232408+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:20:17.232408+0000","last_undegraded":"2024-09-12T07:20:17.232408+0000","last_fullsized":"2024-09-12T07:20:17.232408+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:48:15.796361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060746,"stat_sum":{"num_bytes":9766249,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":1285,"num_read_kb":17355,"num_write":59,"num_write_kb":2111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"55'225","reported_seq":16398,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.242197+0000","last_change":"2024-09-12T07:19:44.482564+0000","last_active":"2024-09-12T07:20:17.242197+0000","last_peered":"2024-09-12T07:20:17.242197+0000","last_clean":"2024-09-12T07:20:17.242197+0000","last_became_active":"2024-09-12T07:19:42.456127+0000","last_became_peered":"2024-09-12T07:19:42.456127+0000","last_unstale":"2024-09-12T07:20:17.242197+0000","last_undegraded":"2024-09-12T07:20:17.242197+0000","last_fullsized":"2024-09-12T07:20:17.242197+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:25:18.311652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793039999999999,"stat_sum":{"num_bytes":36095087,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":16684,"num_read_kb":47561,"num_write":297,"num_write_kb":24067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":560283846,"num_objects":650,"num_object_clones":134,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":650,"num_whiteouts":268,"num_read":111144,"num_read_kb":411035,"num_write":5763,"num_write_kb":282020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":49942512,"num_keys_recovered":48,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434601984,"data_stored":462173036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4721,"ondisk_log_size":4721,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100964,"kb_used_data":69772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94270876,"statfs":{"total":96636764160,"available":96533377024,"internally_reserved":0,"allocated":71446528,"data_stored":73019625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50488,"kb_used_data":19296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321352,"statfs":{"total":96636764160,"available":96585064448,"internally_reserved":0,"allocated":19759104,"data_stored":19378068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100052,"kb_used_data":64708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94271788,"statfs":{"total":96636764160,"available":96534310912,"internally_reserved":0,"allocated":66260992,"data_stored":70451527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542157,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":142824,"kb_used_data":107476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94229016,"statfs":{"total":96636764160,"available":96490512384,"internally_reserved":0,"allocated":110055424,"data_stored":116991307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108832,"kb_used_data":69304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94263008,"statfs":{"total":96636764160,"available":96525320192,"internally_reserved":0,"allocated":70967296,"data_stored":70427411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542158,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44892,"kb_used_data":9536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326948,"statfs":{"total":96636764160,"available":96590794752,"internally_reserved":0,"allocated":9764864,"data_stored":9371476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65360,"kb_used_data":29968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94306480,"statfs":{"total":96636764160,"available":96569835520,"internally_reserved":0,"allocated":30687232,"data_stored":30932502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95940,"kb_used_data":60564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94275900,"statfs":{"total":96636764160,"available":96538521600,"internally_reserved":0,"allocated":62017536,"data_stored":64798491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60780544,"data_stored":64860235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30044160,"data_stored":31584611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9109504,"data_stored":9083549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":70332416,"data_stored":72938934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":109400064,"data_stored":117637258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65613824,"data_stored":73896863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":19103744,"data_stored":19090141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":70217728,"data_stored":73081445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:20:20.034 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-12T07:20:20.034 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:20:20.034 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-12T07:20:20.035 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T07:20:20.080 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:20.080 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:20.089 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:20.132 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:20.133+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:20.196 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:20.196 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:20.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:20.245 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:20.246+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:20.248 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is set 2024-09-12T07:20:20.318 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone17 from 0x7611b7e (overlap 0x2ad260) to 0x0 2024-09-12T07:20:20.326 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:20.328+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:20.359 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:20.359 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:20.368 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:20.390 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-12T07:20:20.396 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:20:20.396 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:20.396 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 60, 2024-09-12T07:20:20.396 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "8412ed58-ac5e-4549-b523-1a58cf9f9429", 2024-09-12T07:20:20.397 INFO:teuthology.orchestra.run.smithi046.stdout: "created": "2024-09-12T07:19:10.328057+0000", 2024-09-12T07:20:20.397 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-12T07:20:20.244783+0000", 2024-09-12T07:20:20.397 INFO:teuthology.orchestra.run.smithi046.stdout: "last_up_change": "2024-09-12T07:19:22.177473+0000", 2024-09-12T07:20:20.397 INFO:teuthology.orchestra.run.smithi046.stdout: "last_in_change": "2024-09-12T07:20:06.162815+0000", 2024-09-12T07:20:20.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T07:20:20.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_num": 5806080, 2024-09-12T07:20:20.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_set": [ 2024-09-12T07:20:20.397 INFO:teuthology.orchestra.run.smithi046.stdout: "nodeep-scrub", 2024-09-12T07:20:20.397 INFO:teuthology.orchestra.run.smithi046.stdout: "noscrub", 2024-09-12T07:20:20.397 INFO:teuthology.orchestra.run.smithi046.stdout: "pglog_hardlimit", 2024-09-12T07:20:20.397 INFO:teuthology.orchestra.run.smithi046.stdout: "purged_snapdirs", 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_deletes", 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "sortbitwise" 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_version": 5, 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_snapshot": "", 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_max": 3, 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd": 8, 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "require_min_compat_client": "luminous", 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "min_compat_client": "jewel", 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "require_osd_release": "squid", 2024-09-12T07:20:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_crimson": false, 2024-09-12T07:20:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "pools": [ 2024-09-12T07:20:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 1, 2024-09-12T07:20:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": ".mgr", 2024-09-12T07:20:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "create_time": "2024-09-12T07:19:23.507151+0000", 2024-09-12T07:20:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T07:20:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_names": "hashpspool", 2024-09-12T07:20:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "type": 1, 2024-09-12T07:20:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 2, 2024-09-12T07:20:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": 1, 2024-09-12T07:20:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_rule": 0, 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_count": 0, 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_target": 0, 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "is_stretch_pool": false, 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "object_hash": 2, 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscale_mode": "off", 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": 1, 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num": 1, 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num_target": 1, 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_target": 1, 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_pending": 1, 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "last_pg_merge_meta": { 2024-09-12T07:20:20.400 INFO:teuthology.orchestra.run.smithi046.stdout: "source_pgid": "0.0", 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: "ready_epoch": 0, 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_started": 0, 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_clean": 0, 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: "source_version": "0'0", 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: "target_version": "0'0" 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: "last_change": "18", 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend": "0", 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: "auid": 0, 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_mode": "selfmanaged", 2024-09-12T07:20:20.401 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_seq": 0, 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_epoch": 0, 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_snaps": [], 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps": "[]", 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_bytes": 0, 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_objects": 0, 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tiers": [], 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tier_of": -1, 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "read_tier": -1, 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "write_tier": -1, 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_mode": "none", 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_bytes": 0, 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_objects": 0, 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T07:20:20.402 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T07:20:20.403 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T07:20:20.403 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_flush_age": 0, 2024-09-12T07:20:20.403 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_evict_age": 0, 2024-09-12T07:20:20.403 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profile": "", 2024-09-12T07:20:20.403 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_params": { 2024-09-12T07:20:20.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "none" 2024-09-12T07:20:20.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.403 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_period": 0, 2024-09-12T07:20:20.403 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_count": 0, 2024-09-12T07:20:20.403 INFO:teuthology.orchestra.run.smithi046.stdout: "use_gmt_hitset": true, 2024-09-12T07:20:20.403 INFO:teuthology.orchestra.run.smithi046.stdout: "min_read_recency_for_promote": 0, 2024-09-12T07:20:20.403 INFO:teuthology.orchestra.run.smithi046.stdout: "min_write_recency_for_promote": 0, 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_search_last_n": 0, 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: "grade_table": [], 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: "stripe_width": 0, 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: "expected_num_objects": 0, 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: "fast_read": false, 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: "options": { 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_max": 32, 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_min": 1 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: "application_metadata": { 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": {} 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.404 INFO:teuthology.orchestra.run.smithi046.stdout: "read_balance": { 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: "score_type": "Fair distribution", 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: "score_acting": 8, 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: "score_stable": 8, 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: "optimal_score": 0.25, 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_acting": 2, 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_stable": 2, 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity_weighted": 0.88148117065429688, 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.405 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 2, 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": "rbd", 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "create_time": "2024-09-12T07:19:28.593632+0000", 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 8193, 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type": 1, 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 2, 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": 1, 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_rule": 0, 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_count": 0, 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_target": 0, 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "is_stretch_pool": false, 2024-09-12T07:20:20.406 INFO:teuthology.orchestra.run.smithi046.stdout: "object_hash": 2, 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscale_mode": "off", 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": 8, 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num": 8, 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num_target": 8, 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_target": 8, 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_pending": 8, 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "last_pg_merge_meta": { 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "source_pgid": "0.0", 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "ready_epoch": 0, 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_started": 0, 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_clean": 0, 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "source_version": "0'0", 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: "target_version": "0'0" 2024-09-12T07:20:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "last_change": "22", 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend": "0", 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "auid": 0, 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_mode": "selfmanaged", 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_seq": 2, 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_epoch": 22, 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_snaps": [], 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps": "[]", 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_bytes": 0, 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_objects": 0, 2024-09-12T07:20:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: "tiers": [], 2024-09-12T07:20:20.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tier_of": -1, 2024-09-12T07:20:20.409 INFO:teuthology.orchestra.run.smithi046.stdout: "read_tier": -1, 2024-09-12T07:20:20.409 INFO:teuthology.orchestra.run.smithi046.stdout: "write_tier": -1, 2024-09-12T07:20:20.409 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_mode": "none", 2024-09-12T07:20:20.409 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_bytes": 0, 2024-09-12T07:20:20.409 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_objects": 0, 2024-09-12T07:20:20.409 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T07:20:20.409 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T07:20:20.409 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T07:20:20.409 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_flush_age": 0, 2024-09-12T07:20:20.409 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_evict_age": 0, 2024-09-12T07:20:20.409 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profile": "", 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_params": { 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "none" 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_period": 0, 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_count": 0, 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: "use_gmt_hitset": true, 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: "min_read_recency_for_promote": 0, 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: "min_write_recency_for_promote": 0, 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_search_last_n": 0, 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: "grade_table": [], 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: "stripe_width": 0, 2024-09-12T07:20:20.410 INFO:teuthology.orchestra.run.smithi046.stdout: "expected_num_objects": 0, 2024-09-12T07:20:20.411 INFO:teuthology.orchestra.run.smithi046.stdout: "fast_read": false, 2024-09-12T07:20:20.411 INFO:teuthology.orchestra.run.smithi046.stdout: "options": {}, 2024-09-12T07:20:20.411 INFO:teuthology.orchestra.run.smithi046.stdout: "application_metadata": { 2024-09-12T07:20:20.411 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd": {} 2024-09-12T07:20:20.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.411 INFO:teuthology.orchestra.run.smithi046.stdout: "read_balance": { 2024-09-12T07:20:20.411 INFO:teuthology.orchestra.run.smithi046.stdout: "score_type": "Fair distribution", 2024-09-12T07:20:20.411 INFO:teuthology.orchestra.run.smithi046.stdout: "score_acting": 1.9900000095367432, 2024-09-12T07:20:20.411 INFO:teuthology.orchestra.run.smithi046.stdout: "score_stable": 1.9900000095367432, 2024-09-12T07:20:20.411 INFO:teuthology.orchestra.run.smithi046.stdout: "optimal_score": 0.87999999523162842, 2024-09-12T07:20:20.411 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_acting": 1.75, 2024-09-12T07:20:20.411 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_stable": 1.75, 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity_weighted": 0.88148117065429688, 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 3, 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": "pool_client.0", 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: "create_time": "2024-09-12T07:19:40.555421+0000", 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 8193, 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type": 1, 2024-09-12T07:20:20.412 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 2, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": 1, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_rule": 0, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_count": 0, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_target": 0, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "is_stretch_pool": false, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "object_hash": 2, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscale_mode": "off", 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": 32, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num": 32, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num_target": 32, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_target": 32, 2024-09-12T07:20:20.413 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_pending": 32, 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: "last_pg_merge_meta": { 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: "source_pgid": "0.0", 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: "ready_epoch": 0, 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_started": 0, 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_clean": 0, 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: "source_version": "0'0", 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: "target_version": "0'0" 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: "last_change": "60", 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend": "0", 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T07:20:20.414 INFO:teuthology.orchestra.run.smithi046.stdout: "auid": 0, 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_mode": "selfmanaged", 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_seq": 20, 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_epoch": 60, 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_snaps": [], 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps": "[]", 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_bytes": 0, 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_objects": 0, 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tiers": [], 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tier_of": -1, 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "read_tier": -1, 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "write_tier": -1, 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_mode": "none", 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_bytes": 0, 2024-09-12T07:20:20.415 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_objects": 0, 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_flush_age": 0, 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_evict_age": 0, 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profile": "", 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_params": { 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "none" 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_period": 0, 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_count": 0, 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "use_gmt_hitset": true, 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "min_read_recency_for_promote": 0, 2024-09-12T07:20:20.416 INFO:teuthology.orchestra.run.smithi046.stdout: "min_write_recency_for_promote": 0, 2024-09-12T07:20:20.417 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T07:20:20.417 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_search_last_n": 0, 2024-09-12T07:20:20.417 INFO:teuthology.orchestra.run.smithi046.stdout: "grade_table": [], 2024-09-12T07:20:20.417 INFO:teuthology.orchestra.run.smithi046.stdout: "stripe_width": 0, 2024-09-12T07:20:20.417 INFO:teuthology.orchestra.run.smithi046.stdout: "expected_num_objects": 0, 2024-09-12T07:20:20.417 INFO:teuthology.orchestra.run.smithi046.stdout: "fast_read": false, 2024-09-12T07:20:20.417 INFO:teuthology.orchestra.run.smithi046.stdout: "options": {}, 2024-09-12T07:20:20.417 INFO:teuthology.orchestra.run.smithi046.stdout: "application_metadata": { 2024-09-12T07:20:20.417 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd": {} 2024-09-12T07:20:20.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: "read_balance": { 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: "score_type": "Fair distribution", 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: "score_acting": 1.75, 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: "score_stable": 1.75, 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: "optimal_score": 1, 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_acting": 1.75, 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_stable": 1.75, 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity_weighted": 0.88148117065429688, 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:20.418 INFO:teuthology.orchestra.run.smithi046.stdout: "osds": [ 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 0, 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "710c25c3-92d3-4b51-a7fc-670879faa69c", 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 33, 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:20.419 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6816", 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6817", 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6818", 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6819", 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.421 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6822", 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6823", 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.422 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6820", 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6821", 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.423 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6817/4081287959", 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6819/4081287959", 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6823/4081287959", 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6821/4081287959", 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 1, 2024-09-12T07:20:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "04479191-c52e-4b60-aa0d-3e911f869062", 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 49, 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6824", 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6825", 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6826", 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6827", 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6830", 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6831", 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6828", 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6829", 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6825/3980488213", 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6827/3980488213", 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3980488213", 2024-09-12T07:20:20.429 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3980488213", 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 2, 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "ba673e51-96aa-424b-a4df-26432d633ebe", 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:20.430 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:20.431 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:20.431 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 46, 2024-09-12T07:20:20.431 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:20.431 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:20.431 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:20.431 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.431 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6800", 2024-09-12T07:20:20.431 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:20.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.432 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6801", 2024-09-12T07:20:20.432 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:20.432 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.432 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.432 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:20.432 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.432 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6802", 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6803", 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.433 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6806", 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6807", 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.435 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6804", 2024-09-12T07:20:20.435 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:20.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.435 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6805", 2024-09-12T07:20:20.435 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:20.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.435 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.435 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6801/656802655", 2024-09-12T07:20:20.436 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6803/656802655", 2024-09-12T07:20:20.436 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6807/656802655", 2024-09-12T07:20:20.436 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6805/656802655", 2024-09-12T07:20:20.436 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:20.436 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:20.436 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:20.436 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.436 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 3, 2024-09-12T07:20:20.436 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "c516e59c-834c-4fb0-9ff8-bcc80b21d28a", 2024-09-12T07:20:20.436 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 46, 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.437 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6808", 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6809", 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.438 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6810", 2024-09-12T07:20:20.439 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:20.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.439 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.439 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6811", 2024-09-12T07:20:20.439 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:20.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.439 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.439 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:20.439 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6814", 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6815", 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:20.440 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6812", 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6813", 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6809/3848260327", 2024-09-12T07:20:20.441 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6811/3848260327", 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6815/3848260327", 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6813/3848260327", 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 4, 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "450a70cd-ebb2-48ed-a766-184f58e7032e", 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:20.442 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 0.051849365234375, 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 0, 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6816", 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:20.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6817", 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6818", 2024-09-12T07:20:20.444 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6819", 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.445 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6822", 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6823", 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.446 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6820", 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6821", 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6817/3464256744", 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6819/3464256744", 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6823/3464256744", 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6821/3464256744", 2024-09-12T07:20:20.447 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 5, 2024-09-12T07:20:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "a4efa333-bf57-45e7-a281-1d3bc9d2e82b", 2024-09-12T07:20:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 46, 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6824", 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6825", 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6826", 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6827", 2024-09-12T07:20:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6830", 2024-09-12T07:20:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6831", 2024-09-12T07:20:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6828", 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6829", 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6825/1789414693", 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6827/1789414693", 2024-09-12T07:20:20.453 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6831/1789414693", 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6829/1789414693", 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 6, 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "bcaad352-f9ee-426c-bcbf-39c8d031aae0", 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:20.454 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 33, 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6800", 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6801", 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6802", 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.457 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6804", 2024-09-12T07:20:20.457 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:20.457 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.457 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.457 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:20.457 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6810", 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6813", 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6806", 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6808", 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6801/2576774646", 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6804/2576774646", 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6813/2576774646", 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6808/2576774646", 2024-09-12T07:20:20.459 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 7, 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "97e5ef36-58a4-4667-9f9e-b8ced5904d69", 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:20.460 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:20.461 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 46, 2024-09-12T07:20:20.461 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:20.461 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:20.461 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:20.461 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.461 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.461 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6803", 2024-09-12T07:20:20.461 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6805", 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6807", 2024-09-12T07:20:20.462 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6809", 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6814", 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:20.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6815", 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6811", 2024-09-12T07:20:20.464 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6812", 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6805/368918691", 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6809/368918691", 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6815/368918691", 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6812/368918691", 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:20.465 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_xinfo": [ 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 0, 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:18.862785+0000", 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:20.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 1, 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.059718+0000", 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 2, 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:20.467 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:18.575165+0000", 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 3, 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:18.847666+0000", 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:20.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 4, 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.471904+0000", 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 5, 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:20.469 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:20.470 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:20.470 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:20.470 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:20.470 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.061712+0000", 2024-09-12T07:20:20.470 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:20.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.470 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 6, 2024-09-12T07:20:20.470 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.541383+0000", 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 7, 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:20.471 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.072272+0000", 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_upmap": [], 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_upmap_items": [], 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_upmap_primaries": [], 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_temp": [], 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_temp": [], 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklist": {}, 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: "range_blocklist": {}, 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profiles": { 2024-09-12T07:20:20.472 INFO:teuthology.orchestra.run.smithi046.stdout: "default": { 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-failure-domain": "osd", 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: "k": "2", 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: "m": "1", 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: "plugin": "jerasure", 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: "technique": "reed_sol_van" 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps_queue": [], 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: "new_removed_snaps": [], 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: "new_purged_snaps": [], 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_node_flags": {}, 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: "device_class_flags": {}, 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_mode": { 2024-09-12T07:20:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_mode_enabled": false, 2024-09-12T07:20:20.474 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_bucket_count": 0, 2024-09-12T07:20:20.474 INFO:teuthology.orchestra.run.smithi046.stdout: "degraded_stretch_mode": 0, 2024-09-12T07:20:20.474 INFO:teuthology.orchestra.run.smithi046.stdout: "recovering_stretch_mode": 0, 2024-09-12T07:20:20.474 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_mode_bucket": 0 2024-09-12T07:20:20.474 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:20.474 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:20.475 INFO:teuthology.orchestra.run.smithi046.stdout:2471 write 0x7da60fe thru 0x7db1560 (0xb463 bytes) 2024-09-12T07:20:20.475 INFO:teuthology.orchestra.run.smithi046.stdout:2472 trunc from 0x7db1561 to 0xb009bf8 2024-09-12T07:20:20.477 INFO:tasks.thrashosds.thrasher:j is {'epoch': 60, 'fsid': '8412ed58-ac5e-4549-b523-1a58cf9f9429', 'created': '2024-09-12T07:19:10.328057+0000', 'modified': '2024-09-12T07:20:20.244783+0000', 'last_up_change': '2024-09-12T07:19:22.177473+0000', 'last_in_change': '2024-09-12T07:20:06.162815+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T07:19:23.507151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8814811706542969, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T07:19:28.593632+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8814811706542969, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T07:19:40.555421+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 20, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8814811706542969, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '710c25c3-92d3-4b51-a7fc-670879faa69c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6817', 'nonce': 4081287959}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6818', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 4081287959}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6822', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6823', 'nonce': 4081287959}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6821', 'nonce': 4081287959}]}, 'public_addr': '172.21.15.46:6817/4081287959', 'cluster_addr': '172.21.15.46:6819/4081287959', 'heartbeat_back_addr': '172.21.15.46:6823/4081287959', 'heartbeat_front_addr': '172.21.15.46:6821/4081287959', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '04479191-c52e-4b60-aa0d-3e911f869062', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6824', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 3980488213}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6827', 'nonce': 3980488213}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3980488213}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6828', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3980488213}]}, 'public_addr': '172.21.15.46:6825/3980488213', 'cluster_addr': '172.21.15.46:6827/3980488213', 'heartbeat_back_addr': '172.21.15.46:6831/3980488213', 'heartbeat_front_addr': '172.21.15.46:6829/3980488213', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'ba673e51-96aa-424b-a4df-26432d633ebe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6801', 'nonce': 656802655}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6802', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 656802655}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6806', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 656802655}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6805', 'nonce': 656802655}]}, 'public_addr': '172.21.15.46:6801/656802655', 'cluster_addr': '172.21.15.46:6803/656802655', 'heartbeat_back_addr': '172.21.15.46:6807/656802655', 'heartbeat_front_addr': '172.21.15.46:6805/656802655', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c516e59c-834c-4fb0-9ff8-bcc80b21d28a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6809', 'nonce': 3848260327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6810', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 3848260327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6814', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 3848260327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6813', 'nonce': 3848260327}]}, 'public_addr': '172.21.15.46:6809/3848260327', 'cluster_addr': '172.21.15.46:6811/3848260327', 'heartbeat_back_addr': '172.21.15.46:6815/3848260327', 'heartbeat_front_addr': '172.21.15.46:6813/3848260327', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '450a70cd-ebb2-48ed-a766-184f58e7032e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.051849365234375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6816', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6817', 'nonce': 3464256744}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6818', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6819', 'nonce': 3464256744}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6822', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6823', 'nonce': 3464256744}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6820', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6821', 'nonce': 3464256744}]}, 'public_addr': '172.21.15.64:6817/3464256744', 'cluster_addr': '172.21.15.64:6819/3464256744', 'heartbeat_back_addr': '172.21.15.64:6823/3464256744', 'heartbeat_front_addr': '172.21.15.64:6821/3464256744', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a4efa333-bf57-45e7-a281-1d3bc9d2e82b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6824', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6825', 'nonce': 1789414693}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6826', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6827', 'nonce': 1789414693}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6830', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6831', 'nonce': 1789414693}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6828', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6829', 'nonce': 1789414693}]}, 'public_addr': '172.21.15.64:6825/1789414693', 'cluster_addr': '172.21.15.64:6827/1789414693', 'heartbeat_back_addr': '172.21.15.64:6831/1789414693', 'heartbeat_front_addr': '172.21.15.64:6829/1789414693', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'bcaad352-f9ee-426c-bcbf-39c8d031aae0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6800', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6801', 'nonce': 2576774646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6802', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6804', 'nonce': 2576774646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6810', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6813', 'nonce': 2576774646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6806', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6808', 'nonce': 2576774646}]}, 'public_addr': '172.21.15.64:6801/2576774646', 'cluster_addr': '172.21.15.64:6804/2576774646', 'heartbeat_back_addr': '172.21.15.64:6813/2576774646', 'heartbeat_front_addr': '172.21.15.64:6808/2576774646', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '97e5ef36-58a4-4667-9f9e-b8ced5904d69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6803', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6805', 'nonce': 368918691}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6807', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6809', 'nonce': 368918691}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6814', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6815', 'nonce': 368918691}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6811', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6812', 'nonce': 368918691}]}, 'public_addr': '172.21.15.64:6805/368918691', 'cluster_addr': '172.21.15.64:6809/368918691', 'heartbeat_back_addr': '172.21.15.64:6815/368918691', 'heartbeat_front_addr': '172.21.15.64:6812/368918691', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:18.862785+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.059718+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:18.575165+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:18.847666+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.471904+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.061712+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.541383+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.072272+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T07:20:20.477 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:20:20.479 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:20.479 INFO:teuthology.orchestra.run.smithi046.stdout:2475 write 0xc4737a8 thru 0xc477ba8 (0x4401 bytes) 2024-09-12T07:20:20.479 INFO:teuthology.orchestra.run.smithi046.stdout:2476 writesame 0x2352c2c thru 0x235a673 data_size 0x68(0x7a48 bytes) 2024-09-12T07:20:20.479 INFO:teuthology.orchestra.run.smithi046.stdout:2479 writesame 0x57f14ed thru 0x57f249a data_size 0x9(0xfae bytes) 2024-09-12T07:20:20.479 INFO:teuthology.orchestra.run.smithi046.stdout:2480 punch from 0xa369fd8 to 0xa3755b7, (0xb5df bytes) 2024-09-12T07:20:20.479 INFO:teuthology.orchestra.run.smithi046.stdout:2481 read 0x62609ae thru 0x626881e (0x7e71 bytes) 2024-09-12T07:20:20.479 INFO:teuthology.orchestra.run.smithi046.stdout:2482 read 0x55809d6 thru 0x5580a91 (0xbc bytes) 2024-09-12T07:20:20.479 INFO:teuthology.orchestra.run.smithi046.stdout:2484 read 0xad73f6 thru 0xae28b9 (0xb4c4 bytes) 2024-09-12T07:20:20.479 INFO:teuthology.orchestra.run.smithi046.stdout:2486 punch from 0x424f4b9 to 0x425ddf8, (0xe93f bytes) 2024-09-12T07:20:20.479 INFO:teuthology.orchestra.run.smithi046.stdout:2487 read 0x9d1d1e0 thru 0x9d26be4 (0x9a05 bytes) 2024-09-12T07:20:20.479 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:20.480 INFO:teuthology.orchestra.run.smithi046.stdout:2492 punch from 0x2601d35 to 0x260373b, (0x1a06 bytes) 2024-09-12T07:20:20.480 INFO:teuthology.orchestra.run.smithi046.stdout:2493 read 0x5149b15 thru 0x514b773 (0x1c5f bytes) 2024-09-12T07:20:20.480 INFO:teuthology.orchestra.run.smithi046.stdout:2496 read 0xa53bb3e thru 0xa547909 (0xbdcc bytes) 2024-09-12T07:20:20.480 INFO:teuthology.orchestra.run.smithi046.stdout:2497 write 0x4ad1dfe thru 0x4ade0f4 (0xc2f7 bytes) 2024-09-12T07:20:20.480 INFO:teuthology.orchestra.run.smithi046.stdout:2498 trunc from 0xc477ba9 to 0xfec77a 2024-09-12T07:20:20.480 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:20.477+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:20.480 INFO:teuthology.orchestra.run.smithi046.stdout:2500 trunc from 0xfec77a to 0x228266f 2024-09-12T07:20:20.483 INFO:teuthology.orchestra.run.smithi046.stdout:2501 punch from 0x26742b to 0x26e55e, (0x7133 bytes) 2024-09-12T07:20:20.483 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:20.483 INFO:teuthology.orchestra.run.smithi046.stdout:2504 read 0x21cef42 thru 0x21d6af8 (0x7bb7 bytes) 2024-09-12T07:20:20.483 INFO:teuthology.orchestra.run.smithi046.stdout:2506 read 0x113c904 thru 0x11424a8 (0x5ba5 bytes) 2024-09-12T07:20:20.483 INFO:teuthology.orchestra.run.smithi046.stdout:2508 write 0xa27a7c thru 0xa28610 (0xb95 bytes) 2024-09-12T07:20:20.483 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:20.483 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:20.492 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:20.500 INFO:teuthology.orchestra.run.smithi046.stdout:2509 write 0xbf044a7 thru 0xbf0c543 (0x809d bytes) 2024-09-12T07:20:20.509 INFO:teuthology.orchestra.run.smithi046.stdout:2510 write 0xc62b867 thru 0xc6355d3 (0x9d6d bytes) 2024-09-12T07:20:20.512 INFO:teuthology.orchestra.run.smithi046.stdout:2513 trunc from 0xc6355d4 to 0xc6c6272 2024-09-12T07:20:20.517 INFO:teuthology.orchestra.run.smithi046.stdout:2514 read 0xa2fb117 thru 0xa302b3c (0x7a26 bytes) 2024-09-12T07:20:20.518 INFO:teuthology.orchestra.run.smithi046.stdout:2516 write 0x4a3b7e1 thru 0x4a45bd1 (0xa3f1 bytes) 2024-09-12T07:20:20.521 INFO:teuthology.orchestra.run.smithi046.stdout:2517 read 0x79b6209 thru 0x79b6c82 (0xa7a bytes) 2024-09-12T07:20:20.521 INFO:teuthology.orchestra.run.smithi046.stdout:2519 write 0x33aa587 thru 0x33b7f16 (0xd990 bytes) 2024-09-12T07:20:20.524 INFO:teuthology.orchestra.run.smithi046.stdout:2521 writesame 0xbec4120 thru 0xbed23ce data_size 0x1(0xe2af bytes) 2024-09-12T07:20:20.556 INFO:teuthology.orchestra.run.smithi046.stdout:2523 write 0xdcb58ca thru 0xdcbf227 (0x995e bytes) 2024-09-12T07:20:20.560 INFO:teuthology.orchestra.run.smithi046.stdout:2525 write 0x94e9af5 thru 0x94f6bd1 (0xd0dd bytes) 2024-09-12T07:20:20.562 INFO:teuthology.orchestra.run.smithi046.stdout:2527 read 0x2c31132 thru 0x2c319e7 (0x8b6 bytes) 2024-09-12T07:20:20.565 INFO:teuthology.orchestra.run.smithi046.stdout:2528 trunc from 0xdcbf228 to 0xb5726f9 2024-09-12T07:20:20.576 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:20.576+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:20.576 INFO:teuthology.orchestra.run.smithi046.stdout:2530 punch from 0x30be500 to 0x30cab69, (0xc669 bytes) 2024-09-12T07:20:20.576 INFO:teuthology.orchestra.run.smithi046.stdout:2531 read 0x6214d7b thru 0x6221ea2 (0xd128 bytes) 2024-09-12T07:20:20.577 INFO:teuthology.orchestra.run.smithi046.stdout:2532 read 0x3acdf5c thru 0x3adc8f6 (0xe99b bytes) 2024-09-12T07:20:20.577 INFO:teuthology.orchestra.run.smithi046.stdout:2533 writesame 0x72a13e4 thru 0x72b0bfa data_size 0x2b(0xf817 bytes) 2024-09-12T07:20:20.577 INFO:teuthology.orchestra.run.smithi046.stdout:2534 writesame 0x9b5be76 thru 0x9b5e1e3 data_size 0x5(0x236e bytes) 2024-09-12T07:20:20.579 INFO:teuthology.orchestra.run.smithi046.stdout:2535 writesame 0xb41a0a7 thru 0xb41b9e1 data_size 0x1(0x193b bytes) 2024-09-12T07:20:20.582 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:20.583 INFO:teuthology.orchestra.run.smithi046.stdout:2541 read 0x96b0aeb thru 0x96b6973 (0x5e89 bytes) 2024-09-12T07:20:20.583 INFO:teuthology.orchestra.run.smithi046.stdout:2542 write 0xb0f684a thru 0xb101996 (0xb14d bytes) 2024-09-12T07:20:20.591 INFO:teuthology.orchestra.run.smithi046.stdout:2543 write 0xc3ed6d5 thru 0xc3f6b23 (0x944f bytes) 2024-09-12T07:20:20.593 INFO:teuthology.orchestra.run.smithi046.stdout:2545 writesame 0x8eb2700 thru 0x8eb74ae data_size 0x7(0x4daf bytes) 2024-09-12T07:20:20.597 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:20.597 INFO:teuthology.orchestra.run.smithi046.stdout:2549 writesame 0x390130a thru 0x3903116 data_size 0x7(0x1e0d bytes) 2024-09-12T07:20:20.600 INFO:teuthology.orchestra.run.smithi046.stdout:2550 punch from 0x3def34e to 0x3dfa8e2, (0xb594 bytes) 2024-09-12T07:20:20.600 INFO:teuthology.orchestra.run.smithi046.stdout:2551 write 0x711d32b thru 0x712d0ef (0xfdc5 bytes) 2024-09-12T07:20:20.602 INFO:teuthology.orchestra.run.smithi046.stdout:2552 read 0x3bb2799 thru 0x3bbe83e (0xc0a6 bytes) 2024-09-12T07:20:20.602 INFO:teuthology.orchestra.run.smithi046.stdout:2553 punch from 0x1ad1956 to 0x1ad6282, (0x492c bytes) 2024-09-12T07:20:20.603 INFO:teuthology.orchestra.run.smithi046.stdout:2555 read 0x1389f41 thru 0x139952b (0xf5eb bytes) 2024-09-12T07:20:20.603 INFO:teuthology.orchestra.run.smithi046.stdout:2557 write 0x96bbdb3 thru 0x96c0faf (0x51fd bytes) 2024-09-12T07:20:20.607 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:20.607 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:20.611 INFO:teuthology.orchestra.run.smithi046.stdout:2558 write 0xdc631a1 thru 0xdc63209 (0x69 bytes) 2024-09-12T07:20:20.614 INFO:teuthology.orchestra.run.smithi046.stdout:2559 read 0x19ae542 thru 0x19bcb44 (0xe603 bytes) 2024-09-12T07:20:20.614 INFO:teuthology.orchestra.run.smithi046.stdout:2560 read 0x73fae9d thru 0x73fd48e (0x25f2 bytes) 2024-09-12T07:20:20.614 INFO:teuthology.orchestra.run.smithi046.stdout:2561 read 0xa7a0e25 thru 0xa7a8962 (0x7b3e bytes) 2024-09-12T07:20:20.616 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:20.620 INFO:teuthology.orchestra.run.smithi046.stdout:2563 write 0xde0addb thru 0xde18f33 (0xe159 bytes) 2024-09-12T07:20:20.621 INFO:teuthology.orchestra.run.smithi046.stdout:2565 read 0x5dbbefe thru 0x5dc41b8 (0x82bb bytes) 2024-09-12T07:20:20.621 INFO:teuthology.orchestra.run.smithi046.stdout:2566 writesame 0x1221d7d thru 0x122cf84 data_size 0x6c(0xb208 bytes) 2024-09-12T07:20:20.624 INFO:teuthology.orchestra.run.smithi046.stdout:2567 writesame 0x386a0c4 thru 0x3879991 data_size 0x1(0xf8ce bytes) 2024-09-12T07:20:20.656 INFO:teuthology.orchestra.run.smithi046.stdout:2568 write 0x35200ff thru 0x352f9a8 (0xf8aa bytes) 2024-09-12T07:20:20.658 INFO:teuthology.orchestra.run.smithi046.stdout:2569 write 0x96a511d thru 0x96af7ba (0xa69e bytes) 2024-09-12T07:20:20.659 INFO:teuthology.orchestra.run.smithi046.stdout:2570 punch from 0x69c9a1a to 0x69cfc59, (0x623f bytes) 2024-09-12T07:20:20.668 INFO:teuthology.orchestra.run.smithi046.stdout:2571 trunc from 0xde18f34 to 0x5c61a0b 2024-09-12T07:20:20.677 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:20.678+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:20.691 INFO:teuthology.orchestra.run.smithi046.stdout:2572 writesame 0x9d2ceac thru 0x9d30f41 data_size 0x7(0x4096 bytes) 2024-09-12T07:20:20.693 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:20.693 INFO:teuthology.orchestra.run.smithi046.stdout:2574 writesame 0x210d6b1 thru 0x2110a7f data_size 0x1(0x33cf bytes) 2024-09-12T07:20:20.699 INFO:teuthology.orchestra.run.smithi046.stdout:2575 read 0x44790d8 thru 0x448608e (0xcfb7 bytes) 2024-09-12T07:20:20.699 INFO:teuthology.orchestra.run.smithi046.stdout:2577 clone 19 order 19 su 32768 sc 7 2024-09-12T07:20:20.765 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:20.765 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:20.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:20.778 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:20.779+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:20.796 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:20:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:20:20.808 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-12T07:20:20.248805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":560874233,"num_objects":654,"num_object_clones":134,"num_object_copies":1308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":654,"num_whiteouts":268,"num_read":111210,"num_read_kb":411090,"num_write":5866,"num_write_kb":283650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":49942512,"num_keys_recovered":48,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4796,"ondisk_log_size":4796,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":709352,"kb_used_data":430624,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754265368,"statfs":{"total":773094113280,"available":772367736832,"internally_reserved":0,"allocated":440958976,"data_stored":455370407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":150503677,"num_objects":246,"num_object_clones":47,"num_object_copies":492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":246,"num_whiteouts":153,"num_read":11529,"num_read_kb":92519,"num_write":1798,"num_write_kb":92725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025563"},"pg_stats":[{"pgid":"3.1f","version":"57'242","reported_seq":1844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.232649+0000","last_change":"2024-09-12T07:19:44.455827+0000","last_active":"2024-09-12T07:20:17.232649+0000","last_peered":"2024-09-12T07:20:17.232649+0000","last_clean":"2024-09-12T07:20:17.232649+0000","last_became_active":"2024-09-12T07:19:42.451212+0000","last_became_peered":"2024-09-12T07:19:42.451212+0000","last_unstale":"2024-09-12T07:20:17.232649+0000","last_undegraded":"2024-09-12T07:20:17.232649+0000","last_fullsized":"2024-09-12T07:20:17.232649+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:57.585258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001023672,"stat_sum":{"num_bytes":10940340,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":2844,"num_read_kb":15275,"num_write":311,"num_write_kb":4391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"57'118","reported_seq":141,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.274011+0000","last_change":"2024-09-12T07:20:07.305432+0000","last_active":"2024-09-12T07:20:17.274011+0000","last_peered":"2024-09-12T07:20:17.274011+0000","last_clean":"2024-09-12T07:20:17.274011+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:20:17.274011+0000","last_undegraded":"2024-09-12T07:20:17.274011+0000","last_fullsized":"2024-09-12T07:20:17.274011+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:58.448307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081935099999999996,"stat_sum":{"num_bytes":28822701,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":16696,"num_read_kb":20572,"num_write":107,"num_write_kb":16946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"57'98","reported_seq":17107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.252898+0000","last_change":"2024-09-12T07:19:44.464376+0000","last_active":"2024-09-12T07:20:17.252898+0000","last_peered":"2024-09-12T07:20:17.252898+0000","last_clean":"2024-09-12T07:20:17.252898+0000","last_became_active":"2024-09-12T07:19:42.451973+0000","last_became_peered":"2024-09-12T07:19:42.451973+0000","last_unstale":"2024-09-12T07:20:17.252898+0000","last_undegraded":"2024-09-12T07:20:17.252898+0000","last_fullsized":"2024-09-12T07:20:17.252898+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:27.976836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036949999999999,"stat_sum":{"num_bytes":26782347,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":16959,"num_read_kb":23181,"num_write":89,"num_write_kb":10322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"57'144","reported_seq":1698,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.358238+0000","last_change":"2024-09-12T07:19:44.464287+0000","last_active":"2024-09-12T07:20:17.358238+0000","last_peered":"2024-09-12T07:20:17.358238+0000","last_clean":"2024-09-12T07:20:17.358238+0000","last_became_active":"2024-09-12T07:19:42.455338+0000","last_became_peered":"2024-09-12T07:19:42.455338+0000","last_unstale":"2024-09-12T07:20:17.358238+0000","last_undegraded":"2024-09-12T07:20:17.358238+0000","last_fullsized":"2024-09-12T07:20:17.358238+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:50.776177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012347180000000001,"stat_sum":{"num_bytes":13823415,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":1857,"num_read_kb":9757,"num_write":182,"num_write_kb":2570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"56'97","reported_seq":12959,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.232468+0000","last_change":"2024-09-12T07:19:44.456652+0000","last_active":"2024-09-12T07:20:17.232468+0000","last_peered":"2024-09-12T07:20:17.232468+0000","last_clean":"2024-09-12T07:20:17.232468+0000","last_became_active":"2024-09-12T07:19:42.454990+0000","last_became_peered":"2024-09-12T07:19:42.454990+0000","last_unstale":"2024-09-12T07:20:17.232468+0000","last_undegraded":"2024-09-12T07:20:17.232468+0000","last_fullsized":"2024-09-12T07:20:17.232468+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:13:53.218197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049866,"stat_sum":{"num_bytes":20900457,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":13242,"num_read_kb":14669,"num_write":88,"num_write_kb":11790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"57'114","reported_seq":1053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.377750+0000","last_change":"2024-09-12T07:19:44.459417+0000","last_active":"2024-09-12T07:20:17.377750+0000","last_peered":"2024-09-12T07:20:17.377750+0000","last_clean":"2024-09-12T07:20:17.377750+0000","last_became_active":"2024-09-12T07:19:42.454620+0000","last_became_peered":"2024-09-12T07:19:42.454620+0000","last_unstale":"2024-09-12T07:20:17.377750+0000","last_undegraded":"2024-09-12T07:20:17.377750+0000","last_fullsized":"2024-09-12T07:20:17.377750+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:06.854794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090501500000000001,"stat_sum":{"num_bytes":13596786,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":880,"num_read_kb":2263,"num_write":108,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"57'243","reported_seq":1072,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.422045+0000","last_change":"2024-09-12T07:20:10.196744+0000","last_active":"2024-09-12T07:20:17.422045+0000","last_peered":"2024-09-12T07:20:17.422045+0000","last_clean":"2024-09-12T07:20:17.422045+0000","last_became_active":"2024-09-12T07:20:10.195312+0000","last_became_peered":"2024-09-12T07:20:10.195312+0000","last_unstale":"2024-09-12T07:20:17.422045+0000","last_undegraded":"2024-09-12T07:20:17.422045+0000","last_fullsized":"2024-09-12T07:20:17.422045+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:13.894288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113156,"stat_sum":{"num_bytes":16795101,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":1503,"num_read_kb":10040,"num_write":354,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1101525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"54'118","reported_seq":282,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.274002+0000","last_change":"2024-09-12T07:20:07.446518+0000","last_active":"2024-09-12T07:20:17.274002+0000","last_peered":"2024-09-12T07:20:17.274002+0000","last_clean":"2024-09-12T07:20:17.274002+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:20:17.274002+0000","last_undegraded":"2024-09-12T07:20:17.274002+0000","last_fullsized":"2024-09-12T07:20:17.274002+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:36:33.763137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000443294,"stat_sum":{"num_bytes":22996397,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":2082,"num_read_kb":16378,"num_write":123,"num_write_kb":5396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"55'60","reported_seq":2447,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.232849+0000","last_change":"2024-09-12T07:19:44.456115+0000","last_active":"2024-09-12T07:20:17.232849+0000","last_peered":"2024-09-12T07:20:17.232849+0000","last_clean":"2024-09-12T07:20:17.232849+0000","last_became_active":"2024-09-12T07:19:42.452617+0000","last_became_peered":"2024-09-12T07:19:42.452617+0000","last_unstale":"2024-09-12T07:20:17.232849+0000","last_undegraded":"2024-09-12T07:20:17.232849+0000","last_fullsized":"2024-09-12T07:20:17.232849+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:25.151293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077251,"stat_sum":{"num_bytes":24337497,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":2312,"num_read_kb":9400,"num_write":63,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"57'59","reported_seq":160,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.317817+0000","last_change":"2024-09-12T07:19:44.463507+0000","last_active":"2024-09-12T07:20:17.317817+0000","last_peered":"2024-09-12T07:20:17.317817+0000","last_clean":"2024-09-12T07:20:17.317817+0000","last_became_active":"2024-09-12T07:19:42.455210+0000","last_became_peered":"2024-09-12T07:19:42.455210+0000","last_unstale":"2024-09-12T07:20:17.317817+0000","last_undegraded":"2024-09-12T07:20:17.317817+0000","last_fullsized":"2024-09-12T07:20:17.317817+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:12.237223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049753700000000004,"stat_sum":{"num_bytes":12309059,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":8,"num_read":25,"num_read_kb":108,"num_write":51,"num_write_kb":4628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"57'86","reported_seq":410,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.309870+0000","last_change":"2024-09-12T07:20:07.261139+0000","last_active":"2024-09-12T07:20:17.309870+0000","last_peered":"2024-09-12T07:20:17.309870+0000","last_clean":"2024-09-12T07:20:17.309870+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:20:17.309870+0000","last_undegraded":"2024-09-12T07:20:17.309870+0000","last_fullsized":"2024-09-12T07:20:17.309870+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:57:02.011265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012225490000000001,"stat_sum":{"num_bytes":7216656,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":487,"num_read_kb":1445,"num_write":72,"num_write_kb":4344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"57'121","reported_seq":3850,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.370365+0000","last_change":"2024-09-12T07:19:44.481796+0000","last_active":"2024-09-12T07:20:17.370365+0000","last_peered":"2024-09-12T07:20:17.370365+0000","last_clean":"2024-09-12T07:20:17.370365+0000","last_became_active":"2024-09-12T07:19:42.452277+0000","last_became_peered":"2024-09-12T07:19:42.452277+0000","last_unstale":"2024-09-12T07:20:17.370365+0000","last_undegraded":"2024-09-12T07:20:17.370365+0000","last_fullsized":"2024-09-12T07:20:17.370365+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:59.618843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012485160000000001,"stat_sum":{"num_bytes":31451428,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":4114,"num_read_kb":49211,"num_write":128,"num_write_kb":23849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.289203+0000","last_change":"2024-09-12T07:19:32.474512+0000","last_active":"2024-09-12T07:20:17.289203+0000","last_peered":"2024-09-12T07:20:17.289203+0000","last_clean":"2024-09-12T07:20:17.289203+0000","last_became_active":"2024-09-12T07:19:30.219990+0000","last_became_peered":"2024-09-12T07:19:30.219990+0000","last_unstale":"2024-09-12T07:20:17.289203+0000","last_undegraded":"2024-09-12T07:20:17.289203+0000","last_fullsized":"2024-09-12T07:20:17.289203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:32.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046339800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"57'261","reported_seq":1203,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.421897+0000","last_change":"2024-09-12T07:19:44.455257+0000","last_active":"2024-09-12T07:20:17.421897+0000","last_peered":"2024-09-12T07:20:17.421897+0000","last_clean":"2024-09-12T07:20:17.421897+0000","last_became_active":"2024-09-12T07:19:42.449730+0000","last_became_peered":"2024-09-12T07:19:42.449730+0000","last_unstale":"2024-09-12T07:20:17.421897+0000","last_undegraded":"2024-09-12T07:20:17.421897+0000","last_fullsized":"2024-09-12T07:20:17.421897+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:18:04.089947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059259600000000003,"stat_sum":{"num_bytes":9633736,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":1792,"num_read_kb":8645,"num_write":308,"num_write_kb":1367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.235223+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:20:17.235223+0000","last_peered":"2024-09-12T07:20:17.235223+0000","last_clean":"2024-09-12T07:20:17.235223+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:20:17.235223+0000","last_undegraded":"2024-09-12T07:20:17.235223+0000","last_fullsized":"2024-09-12T07:20:17.235223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"57'76","reported_seq":685,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.422556+0000","last_change":"2024-09-12T07:20:07.211023+0000","last_active":"2024-09-12T07:20:17.422556+0000","last_peered":"2024-09-12T07:20:17.422556+0000","last_clean":"2024-09-12T07:20:17.422556+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:20:17.422556+0000","last_undegraded":"2024-09-12T07:20:17.422556+0000","last_fullsized":"2024-09-12T07:20:17.422556+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:45.978689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087343100000000003,"stat_sum":{"num_bytes":22331570,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":668,"num_read_kb":11201,"num_write":79,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.242263+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:20:17.242263+0000","last_peered":"2024-09-12T07:20:17.242263+0000","last_clean":"2024-09-12T07:20:17.242263+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:20:17.242263+0000","last_undegraded":"2024-09-12T07:20:17.242263+0000","last_fullsized":"2024-09-12T07:20:17.242263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"55'166","reported_seq":525,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.235066+0000","last_change":"2024-09-12T07:19:44.458925+0000","last_active":"2024-09-12T07:20:17.235066+0000","last_peered":"2024-09-12T07:20:17.235066+0000","last_clean":"2024-09-12T07:20:17.235066+0000","last_became_active":"2024-09-12T07:19:42.452704+0000","last_became_peered":"2024-09-12T07:19:42.452704+0000","last_unstale":"2024-09-12T07:20:17.235066+0000","last_undegraded":"2024-09-12T07:20:17.235066+0000","last_fullsized":"2024-09-12T07:20:17.235066+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:35.846662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045577200000000001,"stat_sum":{"num_bytes":2876510,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":1189,"num_read_kb":1980,"num_write":236,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.234965+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:20:17.234965+0000","last_peered":"2024-09-12T07:20:17.234965+0000","last_clean":"2024-09-12T07:20:17.234965+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:20:17.234965+0000","last_undegraded":"2024-09-12T07:20:17.234965+0000","last_fullsized":"2024-09-12T07:20:17.234965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"57'110","reported_seq":3254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.336522+0000","last_change":"2024-09-12T07:20:07.289858+0000","last_active":"2024-09-12T07:20:17.336522+0000","last_peered":"2024-09-12T07:20:17.336522+0000","last_clean":"2024-09-12T07:20:17.336522+0000","last_became_active":"2024-09-12T07:20:07.183819+0000","last_became_peered":"2024-09-12T07:20:07.183819+0000","last_unstale":"2024-09-12T07:20:17.336522+0000","last_undegraded":"2024-09-12T07:20:17.336522+0000","last_fullsized":"2024-09-12T07:20:17.336522+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:25:17.468030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015134619999999999,"stat_sum":{"num_bytes":44012764,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":3059,"num_read_kb":23450,"num_write":139,"num_write_kb":17370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":3070354,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"57'63","reported_seq":580,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.307609+0000","last_change":"2024-09-12T07:19:44.456465+0000","last_active":"2024-09-12T07:20:17.307609+0000","last_peered":"2024-09-12T07:20:17.307609+0000","last_clean":"2024-09-12T07:20:17.307609+0000","last_became_active":"2024-09-12T07:19:42.454103+0000","last_became_peered":"2024-09-12T07:19:42.454103+0000","last_unstale":"2024-09-12T07:20:17.307609+0000","last_undegraded":"2024-09-12T07:20:17.307609+0000","last_fullsized":"2024-09-12T07:20:17.307609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:11.047251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010980880000000001,"stat_sum":{"num_bytes":7410324,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":14,"num_read":441,"num_read_kb":1640,"num_write":47,"num_write_kb":1264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.252568+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:20:17.252568+0000","last_peered":"2024-09-12T07:20:17.252568+0000","last_clean":"2024-09-12T07:20:17.252568+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:20:17.252568+0000","last_undegraded":"2024-09-12T07:20:17.252568+0000","last_fullsized":"2024-09-12T07:20:17.252568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"53'135","reported_seq":818,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.235066+0000","last_change":"2024-09-12T07:19:44.458868+0000","last_active":"2024-09-12T07:20:17.235066+0000","last_peered":"2024-09-12T07:20:17.235066+0000","last_clean":"2024-09-12T07:20:17.235066+0000","last_became_active":"2024-09-12T07:19:42.451807+0000","last_became_peered":"2024-09-12T07:19:42.451807+0000","last_unstale":"2024-09-12T07:20:17.235066+0000","last_undegraded":"2024-09-12T07:20:17.235066+0000","last_fullsized":"2024-09-12T07:20:17.235066+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.743040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048363399999999998,"stat_sum":{"num_bytes":12607390,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":1548,"num_read_kb":3970,"num_write":144,"num_write_kb":8640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.229409+0000","last_change":"2024-09-12T07:20:07.183543+0000","last_active":"2024-09-12T07:20:17.229409+0000","last_peered":"2024-09-12T07:20:17.229409+0000","last_clean":"2024-09-12T07:20:17.229409+0000","last_became_active":"2024-09-12T07:20:07.183213+0000","last_became_peered":"2024-09-12T07:20:07.183213+0000","last_unstale":"2024-09-12T07:20:17.229409+0000","last_undegraded":"2024-09-12T07:20:17.229409+0000","last_fullsized":"2024-09-12T07:20:17.229409+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.634459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"57'93","reported_seq":1073,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.328840+0000","last_change":"2024-09-12T07:19:44.455560+0000","last_active":"2024-09-12T07:20:17.328840+0000","last_peered":"2024-09-12T07:20:17.328840+0000","last_clean":"2024-09-12T07:20:17.328840+0000","last_became_active":"2024-09-12T07:19:42.451970+0000","last_became_peered":"2024-09-12T07:19:42.451970+0000","last_unstale":"2024-09-12T07:20:17.328840+0000","last_undegraded":"2024-09-12T07:20:17.328840+0000","last_fullsized":"2024-09-12T07:20:17.328840+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:26.326667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083352100000000004,"stat_sum":{"num_bytes":18618824,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":1836,"num_read_kb":10738,"num_write":98,"num_write_kb":15485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.241930+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:20:17.241930+0000","last_peered":"2024-09-12T07:20:17.241930+0000","last_clean":"2024-09-12T07:20:17.241930+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:20:17.241930+0000","last_undegraded":"2024-09-12T07:20:17.241930+0000","last_fullsized":"2024-09-12T07:20:17.241930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"57'623","reported_seq":1113,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.374159+0000","last_change":"2024-09-12T07:20:07.467030+0000","last_active":"2024-09-12T07:20:17.374159+0000","last_peered":"2024-09-12T07:20:17.374159+0000","last_clean":"2024-09-12T07:20:17.374159+0000","last_became_active":"2024-09-12T07:20:07.180577+0000","last_became_peered":"2024-09-12T07:20:07.180577+0000","last_unstale":"2024-09-12T07:20:17.374159+0000","last_undegraded":"2024-09-12T07:20:17.374159+0000","last_fullsized":"2024-09-12T07:20:17.374159+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:47.344526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089638599999999999,"stat_sum":{"num_bytes":15519005,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":3427,"num_read_kb":9930,"num_write":1046,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":261251,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":91,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.252567+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:20:17.252567+0000","last_peered":"2024-09-12T07:20:17.252567+0000","last_clean":"2024-09-12T07:20:17.252567+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:20:17.252567+0000","last_undegraded":"2024-09-12T07:20:17.252567+0000","last_fullsized":"2024-09-12T07:20:17.252567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":181,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.241930+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:20:17.241930+0000","last_peered":"2024-09-12T07:20:17.241930+0000","last_clean":"2024-09-12T07:20:17.241930+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:20:17.241930+0000","last_undegraded":"2024-09-12T07:20:17.241930+0000","last_fullsized":"2024-09-12T07:20:17.241930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"57'114","reported_seq":503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.310049+0000","last_change":"2024-09-12T07:20:07.428482+0000","last_active":"2024-09-12T07:20:17.310049+0000","last_peered":"2024-09-12T07:20:17.310049+0000","last_clean":"2024-09-12T07:20:17.310049+0000","last_became_active":"2024-09-12T07:20:07.180763+0000","last_became_peered":"2024-09-12T07:20:07.180763+0000","last_unstale":"2024-09-12T07:20:17.310049+0000","last_undegraded":"2024-09-12T07:20:17.310049+0000","last_fullsized":"2024-09-12T07:20:17.310049+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:33.693067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047491800000000001,"stat_sum":{"num_bytes":23252826,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":7,"num_read":604,"num_read_kb":8717,"num_write":154,"num_write_kb":14586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":8633990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"57'235","reported_seq":3231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.425624+0000","last_change":"2024-09-12T07:19:44.469508+0000","last_active":"2024-09-12T07:20:17.425624+0000","last_peered":"2024-09-12T07:20:17.425624+0000","last_clean":"2024-09-12T07:20:17.425624+0000","last_became_active":"2024-09-12T07:19:42.450230+0000","last_became_peered":"2024-09-12T07:19:42.450230+0000","last_unstale":"2024-09-12T07:20:17.425624+0000","last_undegraded":"2024-09-12T07:20:17.425624+0000","last_fullsized":"2024-09-12T07:20:17.425624+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:36.565217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076460899999999997,"stat_sum":{"num_bytes":29998209,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":4671,"num_read_kb":25061,"num_write":217,"num_write_kb":1817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"57'97","reported_seq":665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.307244+0000","last_change":"2024-09-12T07:19:44.480981+0000","last_active":"2024-09-12T07:20:17.307244+0000","last_peered":"2024-09-12T07:20:17.307244+0000","last_clean":"2024-09-12T07:20:17.307244+0000","last_became_active":"2024-09-12T07:19:42.452637+0000","last_became_peered":"2024-09-12T07:19:42.452637+0000","last_unstale":"2024-09-12T07:20:17.307244+0000","last_undegraded":"2024-09-12T07:20:17.307244+0000","last_fullsized":"2024-09-12T07:20:17.307244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:58:19.525415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000680892,"stat_sum":{"num_bytes":11712168,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":593,"num_read_kb":7565,"num_write":104,"num_write_kb":4494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"55'127","reported_seq":222,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.274107+0000","last_change":"2024-09-12T07:20:07.379907+0000","last_active":"2024-09-12T07:20:17.274107+0000","last_peered":"2024-09-12T07:20:17.274107+0000","last_clean":"2024-09-12T07:20:17.274107+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:20:17.274107+0000","last_undegraded":"2024-09-12T07:20:17.274107+0000","last_fullsized":"2024-09-12T07:20:17.274107+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:02.142020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054644100000000005,"stat_sum":{"num_bytes":11738737,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":1016,"num_read_kb":4367,"num_write":149,"num_write_kb":9733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"57'75","reported_seq":1633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.371568+0000","last_change":"2024-09-12T07:20:07.249347+0000","last_active":"2024-09-12T07:20:17.371568+0000","last_peered":"2024-09-12T07:20:17.371568+0000","last_clean":"2024-09-12T07:20:17.371568+0000","last_became_active":"2024-09-12T07:20:07.183065+0000","last_became_peered":"2024-09-12T07:20:07.183065+0000","last_unstale":"2024-09-12T07:20:17.371568+0000","last_undegraded":"2024-09-12T07:20:17.371568+0000","last_fullsized":"2024-09-12T07:20:17.371568+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:29.900552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050446300000000003,"stat_sum":{"num_bytes":20312597,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":1481,"num_read_kb":6145,"num_write":69,"num_write_kb":12726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":8578058,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"57'100","reported_seq":808,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.317589+0000","last_change":"2024-09-12T07:19:44.482470+0000","last_active":"2024-09-12T07:20:17.317589+0000","last_peered":"2024-09-12T07:20:17.317589+0000","last_clean":"2024-09-12T07:20:17.317589+0000","last_became_active":"2024-09-12T07:19:42.450577+0000","last_became_peered":"2024-09-12T07:19:42.450577+0000","last_unstale":"2024-09-12T07:20:17.317589+0000","last_undegraded":"2024-09-12T07:20:17.317589+0000","last_fullsized":"2024-09-12T07:20:17.317589+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:25:06.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317836,"stat_sum":{"num_bytes":17183410,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":968,"num_read_kb":9066,"num_write":114,"num_write_kb":2949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"57'274","reported_seq":1350,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.339612+0000","last_change":"2024-09-12T07:19:44.481850+0000","last_active":"2024-09-12T07:20:17.339612+0000","last_peered":"2024-09-12T07:20:17.339612+0000","last_clean":"2024-09-12T07:20:17.339612+0000","last_became_active":"2024-09-12T07:19:42.452236+0000","last_became_peered":"2024-09-12T07:19:42.452236+0000","last_unstale":"2024-09-12T07:20:17.339612+0000","last_undegraded":"2024-09-12T07:20:17.339612+0000","last_fullsized":"2024-09-12T07:20:17.339612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:53.534115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012939710000000001,"stat_sum":{"num_bytes":3004365,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":2844,"num_read_kb":3815,"num_write":371,"num_write_kb":1999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"57'259","reported_seq":804,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.242263+0000","last_change":"2024-09-12T07:19:44.482179+0000","last_active":"2024-09-12T07:20:17.242263+0000","last_peered":"2024-09-12T07:20:17.242263+0000","last_clean":"2024-09-12T07:20:17.242263+0000","last_became_active":"2024-09-12T07:19:42.452155+0000","last_became_peered":"2024-09-12T07:19:42.452155+0000","last_unstale":"2024-09-12T07:20:17.242263+0000","last_undegraded":"2024-09-12T07:20:17.242263+0000","last_fullsized":"2024-09-12T07:20:17.242263+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.797732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011884759999999999,"stat_sum":{"num_bytes":13654463,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":1668,"num_read_kb":5871,"num_write":326,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"57'60","reported_seq":236,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.289322+0000","last_change":"2024-09-12T07:19:44.481020+0000","last_active":"2024-09-12T07:20:17.289322+0000","last_peered":"2024-09-12T07:20:17.289322+0000","last_clean":"2024-09-12T07:20:17.289322+0000","last_became_active":"2024-09-12T07:19:42.450535+0000","last_became_peered":"2024-09-12T07:19:42.450535+0000","last_unstale":"2024-09-12T07:20:17.289322+0000","last_undegraded":"2024-09-12T07:20:17.289322+0000","last_fullsized":"2024-09-12T07:20:17.289322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:49.955754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073032100000000003,"stat_sum":{"num_bytes":5271552,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":107,"num_read_kb":2266,"num_write":59,"num_write_kb":5105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"55'71","reported_seq":2402,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.242160+0000","last_change":"2024-09-12T07:19:44.481275+0000","last_active":"2024-09-12T07:20:17.242160+0000","last_peered":"2024-09-12T07:20:17.242160+0000","last_clean":"2024-09-12T07:20:17.242160+0000","last_became_active":"2024-09-12T07:19:42.452104+0000","last_became_peered":"2024-09-12T07:19:42.452104+0000","last_unstale":"2024-09-12T07:20:17.242160+0000","last_undegraded":"2024-09-12T07:20:17.242160+0000","last_fullsized":"2024-09-12T07:20:17.242160+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:26:53.102880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079083699999999996,"stat_sum":{"num_bytes":15311876,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":2262,"num_read_kb":29393,"num_write":71,"num_write_kb":11641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"54'57","reported_seq":1428,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.232408+0000","last_change":"2024-09-12T07:19:44.456086+0000","last_active":"2024-09-12T07:20:17.232408+0000","last_peered":"2024-09-12T07:20:17.232408+0000","last_clean":"2024-09-12T07:20:17.232408+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:20:17.232408+0000","last_undegraded":"2024-09-12T07:20:17.232408+0000","last_fullsized":"2024-09-12T07:20:17.232408+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:48:15.796361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060746,"stat_sum":{"num_bytes":9766249,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":1285,"num_read_kb":17355,"num_write":59,"num_write_kb":2111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"55'225","reported_seq":16398,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-12T07:20:17.242197+0000","last_change":"2024-09-12T07:19:44.482564+0000","last_active":"2024-09-12T07:20:17.242197+0000","last_peered":"2024-09-12T07:20:17.242197+0000","last_clean":"2024-09-12T07:20:17.242197+0000","last_became_active":"2024-09-12T07:19:42.456127+0000","last_became_peered":"2024-09-12T07:19:42.456127+0000","last_unstale":"2024-09-12T07:20:17.242197+0000","last_undegraded":"2024-09-12T07:20:17.242197+0000","last_fullsized":"2024-09-12T07:20:17.242197+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:25:18.311652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793039999999999,"stat_sum":{"num_bytes":36095087,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":16684,"num_read_kb":47561,"num_write":297,"num_write_kb":24067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":560283846,"num_objects":650,"num_object_clones":134,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":650,"num_whiteouts":268,"num_read":111144,"num_read_kb":411035,"num_write":5763,"num_write_kb":282020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":49942512,"num_keys_recovered":48,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434601984,"data_stored":462173036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4721,"ondisk_log_size":4721,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100964,"kb_used_data":69772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94270876,"statfs":{"total":96636764160,"available":96533377024,"internally_reserved":0,"allocated":71446528,"data_stored":73019625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50488,"kb_used_data":19296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321352,"statfs":{"total":96636764160,"available":96585064448,"internally_reserved":0,"allocated":19759104,"data_stored":19378068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100052,"kb_used_data":64708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94271788,"statfs":{"total":96636764160,"available":96534310912,"internally_reserved":0,"allocated":66260992,"data_stored":70451527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542157,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":142824,"kb_used_data":107476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94229016,"statfs":{"total":96636764160,"available":96490512384,"internally_reserved":0,"allocated":110055424,"data_stored":116991307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108832,"kb_used_data":69304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94263008,"statfs":{"total":96636764160,"available":96525320192,"internally_reserved":0,"allocated":70967296,"data_stored":70427411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542158,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44892,"kb_used_data":9536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326948,"statfs":{"total":96636764160,"available":96590794752,"internally_reserved":0,"allocated":9764864,"data_stored":9371476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65360,"kb_used_data":29968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94306480,"statfs":{"total":96636764160,"available":96569835520,"internally_reserved":0,"allocated":30687232,"data_stored":30932502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95940,"kb_used_data":60564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94275900,"statfs":{"total":96636764160,"available":96538521600,"internally_reserved":0,"allocated":62017536,"data_stored":64798491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60780544,"data_stored":64860235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30044160,"data_stored":31584611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9109504,"data_stored":9083549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":70332416,"data_stored":72938934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":109400064,"data_stored":117637258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65613824,"data_stored":73896863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":19103744,"data_stored":19090141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":70217728,"data_stored":73081445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:20:20.810 INFO:tasks.thrashosds.thrasher:Setting 2.7 to [6, 7] 2024-09-12T07:20:20.810 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.7', '6', '7'] 2024-09-12T07:20:20.811 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.7 6 7 2024-09-12T07:20:20.879 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:20.880+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:20.882 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:20.882 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:20.891 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:20.980 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:20.982+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:21.041 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:21.041 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:21.050 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:21.091 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:21.093+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:21.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:21.182 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:21.183+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:21.199 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:21.199 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:21.208 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:set 2.7 pg_upmap mapping to [6,7] 2024-09-12T07:20:21.265 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:21.265 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:21.283 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:21.283+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:21.283 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:21.317 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:21.335 INFO:teuthology.orchestra.run.smithi046.stdout:flattening image image_client.0-clone18 2024-09-12T07:20:21.335 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:21.336 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:21.354 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:21.383 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:21.385+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:21.386 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:21.386 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:21.393 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:21.393 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:21.393 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:21.393 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:21.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:21.412 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:21.439 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:21.457 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-12T07:20:21.458 INFO:teuthology.orchestra.run.smithi046.stdout:2578 punch from 0x6b7969d to 0x6b854e3, (0xbe46 bytes) 2024-09-12T07:20:21.461 INFO:teuthology.orchestra.run.smithi046.stdout:2579 punch from 0x2092572 to 0x2093e9b, (0x1929 bytes) 2024-09-12T07:20:21.461 INFO:teuthology.orchestra.run.smithi046.stdout:2580 read 0x2b9611f thru 0x2b98889 (0x276b bytes) 2024-09-12T07:20:21.462 INFO:teuthology.orchestra.run.smithi046.stdout:2581 read 0x6a88c94 thru 0x6a979eb (0xed58 bytes) 2024-09-12T07:20:21.470 INFO:teuthology.orchestra.run.smithi046.stdout:2582 trunc from 0x9d30f42 to 0x290879a 2024-09-12T07:20:21.483 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:21.484+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:21.499 INFO:teuthology.orchestra.run.smithi046.stdout:2583 read 0x237f746 thru 0x2385808 (0x60c3 bytes) 2024-09-12T07:20:21.510 INFO:teuthology.orchestra.run.smithi046.stdout:2586 write 0x6f46f88 thru 0x6f55004 (0xe07d bytes) 2024-09-12T07:20:21.519 INFO:teuthology.orchestra.run.smithi046.stdout:2587 read 0x5cc712c thru 0x5cd663c (0xf511 bytes) 2024-09-12T07:20:21.519 INFO:teuthology.orchestra.run.smithi046.stdout:2590 read 0x659ee93 thru 0x65a920c (0xa37a bytes) 2024-09-12T07:20:21.519 INFO:teuthology.orchestra.run.smithi046.stdout:2591 punch from 0x6ac0369 to 0x6ac877a, (0x8411 bytes) 2024-09-12T07:20:21.519 INFO:teuthology.orchestra.run.smithi046.stdout:2592 write 0x5b97044 thru 0x5b9b943 (0x4900 bytes) 2024-09-12T07:20:21.526 INFO:teuthology.orchestra.run.smithi046.stdout:2595 write 0x41c47b9 thru 0x41c4de4 (0x62c bytes) 2024-09-12T07:20:21.537 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:21.537 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:21.543 INFO:teuthology.orchestra.run.smithi046.stdout:2596 write 0xeb32084 thru 0xeb3e62e (0xc5ab bytes) 2024-09-12T07:20:21.546 INFO:teuthology.orchestra.run.smithi046.stdout:2598 writesame 0x2dd2e25 thru 0x2ddaf7d data_size 0x1(0x8159 bytes) 2024-09-12T07:20:21.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:21.558 INFO:teuthology.orchestra.run.smithi046.stdout:2599 write 0x5b63160 thru 0x5b6a7e6 (0x7687 bytes) 2024-09-12T07:20:21.569 INFO:teuthology.orchestra.run.smithi046.stdout:2601 trunc from 0xeb3e62f to 0x7e5af52 2024-09-12T07:20:21.578 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:21.578 INFO:teuthology.orchestra.run.smithi046.stdout:2603 writesame 0x2b171cc thru 0x2b1c383 data_size 0x14(0x51b8 bytes) 2024-09-12T07:20:21.583 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:21.583 INFO:teuthology.orchestra.run.smithi046.stdout:2607 punch from 0x2821b42 to 0x282cad2, (0xaf90 bytes) 2024-09-12T07:20:21.583 INFO:teuthology.orchestra.run.smithi046.stdout:2608 punch from 0x6845d71 to 0x684a308, (0x4597 bytes) 2024-09-12T07:20:21.583 INFO:teuthology.orchestra.run.smithi046.stdout:2609 write 0x6b3253e thru 0x6b34828 (0x22eb bytes) 2024-09-12T07:20:21.584 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:21.584+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:21.585 INFO:teuthology.orchestra.run.smithi046.stdout:2610 write 0x330f49f thru 0x331e214 (0xed76 bytes) 2024-09-12T07:20:21.599 INFO:teuthology.orchestra.run.smithi046.stdout:2612 write 0xadb50b5 thru 0xadbbea3 (0x6def bytes) 2024-09-12T07:20:21.602 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:21.602 INFO:teuthology.orchestra.run.smithi046.stdout:2615 read 0x75b35f3 thru 0x75b736f (0x3d7d bytes) 2024-09-12T07:20:21.602 INFO:teuthology.orchestra.run.smithi046.stdout:2617 write 0x26edade thru 0x26ee7bb (0xcde bytes) 2024-09-12T07:20:21.612 INFO:teuthology.orchestra.run.smithi046.stdout:2618 write 0xdfc1bcb thru 0xdfd06a3 (0xead9 bytes) 2024-09-12T07:20:21.616 INFO:teuthology.orchestra.run.smithi046.stdout:2619 write 0x3099254 thru 0x30a2b7c (0x9929 bytes) 2024-09-12T07:20:21.620 INFO:teuthology.orchestra.run.smithi046.stdout:2620 writesame 0x190f46a thru 0x191aa4e data_size 0x43(0xb5e5 bytes) 2024-09-12T07:20:21.625 INFO:teuthology.orchestra.run.smithi046.stdout:2621 clone 20 order 20 su 131072 sc 3 2024-09-12T07:20:21.676 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:21.676 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:21.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:21.676 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:21.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.677 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:996 3.0 3:05cac305:::rbd_data.116011b0bca6.0000000000000065:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:20:21.677 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.308560+0000", 2024-09-12T07:20:21.677 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.368398917, 2024-09-12T07:20:21.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014904776, 2024-09-12T07:20:21.677 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:21.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.677 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 996 2024-09-12T07:20:21.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308560+0000", 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308560+0000", 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308561+0000", 2024-09-12T07:20:21.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.536e-06 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308570+0000", 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1090000000000004e-06 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308584+0000", 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3893000000000001e-05 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308592+0000", 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1009999999999992e-06 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308644+0000", 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2117999999999998e-05 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.308754+0000", 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109666 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.310773+0000", 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020187310000000002 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.323365+0000", 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012591836 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.323377+0000", 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2109999999999999e-05 2024-09-12T07:20:21.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.323413+0000", 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6245999999999998e-05 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.323464+0000", 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1430000000000001e-05 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.293570673000001, 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:20:21.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:20:21.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:20:21.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66600 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.389749+0000", 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.287209931, 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032114065999999997, 2024-09-12T07:20:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66600 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389749+0000", 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389749+0000", 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389750+0000", 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.421e-06 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389755+0000", 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8250000000000004e-06 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389770+0000", 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5659e-05 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389779+0000", 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4440000000000008e-06 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.401597+0000", 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011818265 2024-09-12T07:20:21.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.401696+0000", 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9097000000000003e-05 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.402026+0000", 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032977499999999998 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.413052+0000", 2024-09-12T07:20:21.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011026094 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421766+0000", 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087136069999999999 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421822+0000", 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6332000000000002e-05 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421863+0000", 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0547000000000003e-05 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66677 3.0 3:07537075:::rbd_data.1160f022dd61.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.389797+0000", 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.287161414, 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022850626999999998, 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.694 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66677 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389797+0000", 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389797+0000", 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389798+0000", 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0999999999999997e-07 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389802+0000", 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.817e-06 2024-09-12T07:20:21.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389809+0000", 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4900000000000003e-06 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389815+0000", 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1099999999999999e-06 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.401560+0000", 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011744409000000001 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.401777+0000", 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000217615 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.402080+0000", 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030243799999999998 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.404470+0000", 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0023900010000000001 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412521+0000", 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080516579999999997 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412587+0000", 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5522e-05 2024-09-12T07:20:21.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412648+0000", 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0757000000000001e-05 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.230492409, 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:20:21.700 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:20:21.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:20:21.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:21.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:20:21.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:20:21.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:21.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:20:21.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:20:21.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:21.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:20:21.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:21.706 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.226787324, 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:21.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:20:21.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:20:21.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:69559 3.1a 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.681624+0000", 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.995334402000001, 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023561162, 2024-09-12T07:20:21.712 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 69559 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.681624+0000", 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.681624+0000", 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.681626+0000", 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5409999999999999e-06 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683088+0000", 2024-09-12T07:20:21.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001462009 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683122+0000", 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683133+0000", 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1559000000000001e-05 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683225+0000", 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2330000000000005e-05 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683325+0000", 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9513999999999996e-05 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683782+0000", 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00045661299999999997 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.691512+0000", 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0077308530000000002 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.705112+0000", 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013599862000000001 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.705155+0000", 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2988999999999998e-05 2024-09-12T07:20:21.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.705185+0000", 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.995e-05 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:69560 3.1a 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.683195+0000", 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.993763163000001, 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024920327999999999, 2024-09-12T07:20:21.718 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 69560 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683195+0000", 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683195+0000", 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683197+0000", 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.624e-06 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683202+0000", 2024-09-12T07:20:21.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1189999999999996e-06 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683892+0000", 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00068997999999999998 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.683905+0000", 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3066e-05 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.687754+0000", 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038485139999999999 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.687860+0000", 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106712 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.688125+0000", 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026430699999999998 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:21.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.692505+0000", 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0043800030000000004 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.708032+0000", 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01552693 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.708079+0000", 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7775999999999998e-05 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.708116+0000", 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6297000000000002e-05 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:69639 3.1a 3:5ec4a676:::rbd_data.1160f022dd61.0000000000000019:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.685937+0000", 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.991021876000001, 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022266522, 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 69639 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685937+0000", 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685937+0000", 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685938+0000", 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02e-06 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685951+0000", 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3621000000000001e-05 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685961+0000", 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0210999999999999e-05 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.685968+0000", 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9040000000000003e-06 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.691551+0000", 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055829089999999996 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.691668+0000", 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117186 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.708203+0000", 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016534671000000001 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78042 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:b [read 8323072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528554+0000", 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.148404105000001, 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011297491, 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78042 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528554+0000", 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528554+0000", 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528555+0000", 2024-09-12T07:20:21.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4200000000000001e-07 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528557+0000", 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0480000000000001e-06 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528565+0000", 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2490000000000001e-06 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528578+0000", 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2513e-05 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539560+0000", 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010982228 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539642+0000", 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1780000000000006e-05 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539852+0000", 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021023099999999999 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78043 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 7798784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528586+0000", 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.148372859999998, 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011197245999999999, 2024-09-12T07:20:21.733 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78043 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528586+0000", 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528586+0000", 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528586+0000", 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9200000000000001e-07 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528588+0000", 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2709999999999999e-06 2024-09-12T07:20:21.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528597+0000", 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4130000000000006e-06 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528602+0000", 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2950000000000001e-06 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539546+0000", 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010943575000000001 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539626+0000", 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0531999999999994e-05 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539783+0000", 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000156668 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78044 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 7929856~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528609+0000", 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.148348999, 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011471324, 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78044 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528609+0000", 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528609+0000", 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528610+0000", 2024-09-12T07:20:21.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2199999999999999e-07 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528612+0000", 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.888e-06 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528620+0000", 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3140000000000004e-06 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528625+0000", 2024-09-12T07:20:21.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.87e-06 2024-09-12T07:20:21.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539809+0000", 2024-09-12T07:20:21.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011183659 2024-09-12T07:20:21.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.539865+0000", 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6563e-05 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540081+0000", 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000215608 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78045 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 8060928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528631+0000", 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.148327099999999, 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011695537000000001, 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78045 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528631+0000", 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528631+0000", 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528632+0000", 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.46e-07 2024-09-12T07:20:21.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528634+0000", 2024-09-12T07:20:21.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9700000000000002e-06 2024-09-12T07:20:21.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528640+0000", 2024-09-12T07:20:21.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7329999999999997e-06 2024-09-12T07:20:21.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528646+0000", 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1699999999999996e-06 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540111+0000", 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011465444999999999 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540164+0000", 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2927999999999999e-05 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540327+0000", 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016294500000000001 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78046 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 8192000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528655+0000", 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.148303468999998, 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012006785000000001, 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78046 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528655+0000", 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528655+0000", 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528655+0000", 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9999999999999998e-07 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528657+0000", 2024-09-12T07:20:21.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.125e-06 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528664+0000", 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5660000000000003e-06 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528668+0000", 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2570000000000001e-06 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540386+0000", 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011717627 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540470+0000", 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3891e-05 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540662+0000", 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000191919 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78047 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:b [read 8323072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.528677+0000", 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.148281441000002, 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012383072, 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78047 2024-09-12T07:20:21.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528677+0000", 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528677+0000", 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528677+0000", 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.65e-07 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528680+0000", 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4449999999999999e-06 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528687+0000", 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5329999999999997e-06 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.528692+0000", 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540693+0000", 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012000613 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.540781+0000", 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8178999999999997e-05 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.541060+0000", 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027922400000000001 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78287 3.19 3:99bc29a5:::rbd_data.116087d6d6a2.0000000000000047:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.635267+0000", 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.041691062000002, 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022254682000000001, 2024-09-12T07:20:21.756 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78287 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635267+0000", 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635267+0000", 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635269+0000", 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.178e-06 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635273+0000", 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.121e-06 2024-09-12T07:20:21.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635281+0000", 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5779999999999999e-06 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635286+0000", 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8199999999999996e-06 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635352+0000", 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5571e-05 2024-09-12T07:20:21.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635473+0000", 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000121795 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.639340+0000", 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038665679999999999 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:21.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.657349+0000", 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018008941000000001 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.657402+0000", 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3142e-05 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.657466+0000", 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4325000000000005e-05 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.657522+0000", 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5643000000000002e-05 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:78288 3.1a 3:58d7858f:::rbd_data.116087d6d6a2.0000000000000048:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.635707+0000", 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.041251171999999, 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016418782, 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 78288 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635707+0000", 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635707+0000", 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635708+0000", 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8100000000000002e-07 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635712+0000", 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635721+0000", 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6539999999999994e-06 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635726+0000", 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5580000000000002e-06 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635768+0000", 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1897999999999998e-05 2024-09-12T07:20:21.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.635844+0000", 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6446999999999995e-05 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.638143+0000", 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022985079999999999 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.651990+0000", 2024-09-12T07:20:21.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013846712000000001 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.652009+0000", 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9548e-05 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.652070+0000", 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0523999999999999e-05 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.652126+0000", 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6462000000000003e-05 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.228441045, 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:20:21.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:20:21.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:88881 3.1a 3:5dc3e11f:::rbd_data.11607a56846.0000000000000016:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 2129920~15013] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:13.612488+0000", 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.064470086, 2024-09-12T07:20:21.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0163359, 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 88881 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.612488+0000", 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.612488+0000", 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.612492+0000", 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7139999999999999e-06 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616402+0000", 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039097059999999998 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616427+0000", 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5667999999999999e-05 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616440+0000", 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2495e-05 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616525+0000", 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5442e-05 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.616693+0000", 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000167691 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.617037+0000", 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034386099999999999 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.627285+0000", 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010248554999999999 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.628694+0000", 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001408379 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.628766+0000", 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1766000000000004e-05 2024-09-12T07:20:21.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.628824+0000", 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8622999999999998e-05 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:97141 3.6 e57/24 3:67e19a26:::rbd_data.1160b5642b3d.000000000000000b:head v 57'262, mlcod=57'260)", 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.666167+0000", 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0107913609999999, 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016803757999999999, 2024-09-12T07:20:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.666167+0000", 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.666167+0000", 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.666169+0000", 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.564e-06 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.669447+0000", 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0032787990000000002 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.669468+0000", 2024-09-12T07:20:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0285e-05 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.669477+0000", 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3570000000000004e-06 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.669553+0000", 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6026999999999996e-05 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.669578+0000", 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5320000000000002e-05 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.682897+0000", 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013318919 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.682971+0000", 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3486999999999996e-05 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.784 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:21.784 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:21.784 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:21.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:21.786 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:21.787+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:21.885 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:21.887+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:21.934 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:21.934 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:21.934 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:21.934 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:21.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:21.987 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:21.988+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:22.049 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:22.049 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:22.058 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:22.102 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:22.104+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:22.167 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77440 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13156352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.922553542999999, 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051399724000000001, 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77440 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246267+0000", 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082256199999999999 2024-09-12T07:20:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246271+0000", 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6700000000000002e-06 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246283+0000", 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.207e-05 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246291+0000", 2024-09-12T07:20:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296645+0000", 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050353872000000001 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296729+0000", 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4215000000000006e-05 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296844+0000", 2024-09-12T07:20:22.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114626 2024-09-12T07:20:22.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.172 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.172 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77446 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.172 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:22.172 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.921555028, 2024-09-12T07:20:22.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051543643, 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77446 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246579+0000", 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013651399999999999 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246583+0000", 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1509999999999997e-06 2024-09-12T07:20:22.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246593+0000", 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5670000000000001e-06 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246599+0000", 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5189999999999998e-06 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297826+0000", 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051226381000000001 2024-09-12T07:20:22.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297903+0000", 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6710000000000002e-05 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297986+0000", 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3801000000000006e-05 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77447 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 5783552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.921387414000002, 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051512637, 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77447 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:22.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:22.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246812+0000", 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000201506 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246817+0000", 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246828+0000", 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1693e-05 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246836+0000", 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4429999999999997e-06 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298021+0000", 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051184940999999998 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298067+0000", 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6292000000000003e-05 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298123+0000", 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5828999999999997e-05 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77448 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.921147676, 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051506024999999997, 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77448 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246892+0000", 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1854999999999998e-05 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246895+0000", 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3129999999999999e-06 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246904+0000", 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6179999999999995e-06 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246911+0000", 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7939999999999997e-06 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298164+0000", 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051253559999999997 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298249+0000", 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4549000000000007e-05 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298356+0000", 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107336 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77449 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:22.185 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.92107648, 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051582830000000003, 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77449 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:22.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247137+0000", 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021599499999999999 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247142+0000", 2024-09-12T07:20:22.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247154+0000", 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1878e-05 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247168+0000", 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4114000000000001e-05 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298391+0000", 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051223249999999998 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298446+0000", 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4366000000000001e-05 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298504+0000", 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8273000000000003e-05 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77450 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12681216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.920816909999999, 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051562375000000001, 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77450 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2799999999999996e-07 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247184+0000", 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247192+0000", 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5920000000000003e-06 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247197+0000", 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6720000000000001e-06 2024-09-12T07:20:22.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298550+0000", 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051352082 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298641+0000", 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1236000000000005e-05 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298743+0000", 2024-09-12T07:20:22.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102428 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77451 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7061504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.920792796000001, 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051813773, 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77451 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247335+0000", 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013012599999999999 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247340+0000", 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.544e-06 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247351+0000", 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.111e-05 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247358+0000", 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9859999999999997e-06 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298909+0000", 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051551781999999997 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298960+0000", 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0594999999999999e-05 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299019+0000", 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8629999999999999e-05 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77452 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.920628015999998, 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051892134999999999, 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77452 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:22.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247535+0000", 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165508 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247552+0000", 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6514000000000001e-05 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247565+0000", 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3372e-05 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247577+0000", 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299064+0000", 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051486697999999997 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299150+0000", 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6549000000000001e-05 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299262+0000", 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011135 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77453 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.920407295, 2024-09-12T07:20:22.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051901623000000001, 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77453 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247710+0000", 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011951 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247713+0000", 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4539999999999999e-06 2024-09-12T07:20:22.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247720+0000", 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0339999999999999e-06 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247730+0000", 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0030000000000001e-05 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299297+0000", 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051566446000000002 2024-09-12T07:20:22.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299382+0000", 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4963000000000006e-05 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299492+0000", 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011018600000000001 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.206 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77454 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7618560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.920258747999998, 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051921851999999998, 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77454 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247900+0000", 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016088699999999999 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247904+0000", 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6950000000000002e-06 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247912+0000", 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7129999999999994e-06 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247921+0000", 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0299999999999999e-06 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299526+0000", 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051604744000000001 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299601+0000", 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5129000000000005e-05 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299661+0000", 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9654000000000002e-05 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77455 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12746752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.920064879999998, 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051971254000000001, 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77455 2024-09-12T07:20:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248181+0000", 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024790000000000001 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248185+0000", 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5079999999999999e-06 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248197+0000", 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.132e-05 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248203+0000", 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6320000000000002e-06 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299705+0000", 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051501342999999998 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299799+0000", 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4340999999999997e-05 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299904+0000", 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010521 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77456 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7979008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.919782654999999, 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051860577999999997, 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77456 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248545+0000", 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033009700000000001 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248550+0000", 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248560+0000", 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248568+0000", 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4939999999999997e-06 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299973+0000", 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051405470000000002 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300019+0000", 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5448999999999998e-05 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300076+0000", 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7073000000000001e-05 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77457 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13172736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:22.219 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.919416327, 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051737914000000003, 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77457 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248582+0000", 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248584+0000", 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248593+0000", 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248609+0000", 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5664000000000001e-05 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300117+0000", 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051507605999999997 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300212+0000", 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4919000000000004e-05 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300319+0000", 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010776399999999999 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.223 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77458 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.919373909000001, 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051863578, 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77458 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248627+0000", 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1389999999999999e-06 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248635+0000", 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4019999999999996e-06 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248641+0000", 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1689999999999997e-06 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300359+0000", 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051717889000000003 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300418+0000", 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8689e-05 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300487+0000", 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9756000000000001e-05 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77459 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13238272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.919348215999999, 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052076504000000003, 2024-09-12T07:20:22.228 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77459 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250611+0000", 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019617879999999999 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250618+0000", 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1829999999999997e-06 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250629+0000", 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0438000000000001e-05 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250648+0000", 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9355e-05 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300528+0000", 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049879252999999998 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300619+0000", 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1910000000000006e-05 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300726+0000", 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010657700000000001 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77467 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.232 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.917142396999999, 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05160754, 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77467 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250856+0000", 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250858+0000", 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9180000000000001e-06 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250864+0000", 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.066e-06 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250869+0000", 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302283+0000", 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051414591000000003 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302378+0000", 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4283000000000005e-05 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302463+0000", 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5249999999999999e-05 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.236 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77468 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14909440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.917122824, 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051742162000000001, 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77468 2024-09-12T07:20:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:22.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250877+0000", 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.815e-06 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250883+0000", 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250888+0000", 2024-09-12T07:20:22.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302496+0000", 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051607394000000001 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302553+0000", 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6971000000000001e-05 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302617+0000", 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4418999999999994e-05 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77469 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14974976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.917102645, 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965694, 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.241 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77469 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250897+0000", 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9130000000000001e-06 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250903+0000", 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0129999999999999e-06 2024-09-12T07:20:22.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250908+0000", 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9679999999999997e-06 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302661+0000", 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051753066 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302755+0000", 2024-09-12T07:20:22.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3376999999999997e-05 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302861+0000", 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105924 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77470 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12828672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.917082967999999, 2024-09-12T07:20:22.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052098589000000001, 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77470 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250917+0000", 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8190000000000001e-06 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250923+0000", 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250928+0000", 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9420000000000003e-06 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302894+0000", 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965863000000001 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302955+0000", 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1235999999999994e-05 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303013+0000", 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8167000000000001e-05 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.249 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77471 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12894208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.917063421999998, 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052326263999999997, 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77471 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256961+0000", 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060271209999999999 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256972+0000", 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0755e-05 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256986+0000", 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845e-05 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256996+0000", 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303060+0000", 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.046063661999999998 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303149+0000", 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8949999999999994e-05 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303261+0000", 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011195 2024-09-12T07:20:22.253 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.254 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.254 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.254 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:22.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:22.256 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:22.256+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:22.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:20:22.323 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone19 from 0xdfd06a4 (overlap 0x290879a) to 0xc4f57b 2024-09-12T07:20:22.364 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:22.366+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:22.446 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:101370 3.18 3:1cc04ae3:::rbd_data.11606e757b33.0000000000000016:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 524288 write_size 524288,truncate 292219] snapc 16=[16] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-12T07:20:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:22.446211+0000", 2024-09-12T07:20:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0016705050000000001, 2024-09-12T07:20:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001698024, 2024-09-12T07:20:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 101370 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.446211+0000", 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.446211+0000", 2024-09-12T07:20:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.446214+0000", 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5320000000000001e-06 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.446223+0000", 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4590000000000004e-06 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.446245+0000", 2024-09-12T07:20:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1243e-05 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.446255+0000", 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0118e-05 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.446330+0000", 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5097000000000003e-05 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.446476+0000", 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014634199999999999 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.446668+0000", 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019199899999999999 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.447495+0000", 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000826963 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.447801+0000", 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030567099999999998 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.447861+0000", 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0328999999999998e-05 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.447913+0000", 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1900000000000001e-05 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:101370 3.18 e62/46 ondisk, result = 0)", 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:22.447720+0000", 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.00016146500000000001, 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029015399999999999, 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.447720+0000", 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.447720+0000", 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.447721+0000", 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4499999999999996e-07 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.447726+0000", 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0220000000000004e-06 2024-09-12T07:20:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.447741+0000", 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.438e-05 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.447749+0000", 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9920000000000007e-06 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.447789+0000", 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0326e-05 2024-09-12T07:20:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:22.447796+0000", 2024-09-12T07:20:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9149999999999999e-06 2024-09-12T07:20:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 2 2024-09-12T07:20:22.455 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:22.456 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:22.458+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:22.457 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:22.500 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-12T07:20:22.557 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:22.558+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:22.615 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:22.616 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:22.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:22.665 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:22.667+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:22.675 INFO:teuthology.orchestra.run.smithi046.stdout:2622 write 0x366edf5 thru 0x367503a (0x6246 bytes) 2024-09-12T07:20:22.682 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:22.682 INFO:teuthology.orchestra.run.smithi046.stdout:2625 read 0x5bc77fa thru 0x5bd50cd (0xd8d4 bytes) 2024-09-12T07:20:22.682 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:22.694 INFO:teuthology.orchestra.run.smithi046.stdout:2628 trunc from 0xdfd06a4 to 0x39dbc94 2024-09-12T07:20:22.704 INFO:teuthology.orchestra.run.smithi046.stdout:2629 write 0x347e771 thru 0x3482516 (0x3da6 bytes) 2024-09-12T07:20:22.719 INFO:teuthology.orchestra.run.smithi046.stdout:2630 write 0x6171266 thru 0x6180aed (0xf888 bytes) 2024-09-12T07:20:22.726 INFO:teuthology.orchestra.run.smithi046.stdout:2631 read 0x1e832dc thru 0x1e8786b (0x4590 bytes) 2024-09-12T07:20:22.735 INFO:teuthology.orchestra.run.smithi046.stdout:2632 writesame 0x8f69cca thru 0x8f6fd2f data_size 0x12(0x6066 bytes) 2024-09-12T07:20:22.735 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:22.735 INFO:teuthology.orchestra.run.smithi046.stdout:2634 write 0x7d1b301 thru 0x7d20264 (0x4f64 bytes) 2024-09-12T07:20:22.743 INFO:teuthology.orchestra.run.smithi046.stdout:2635 read 0x70a1d3d thru 0x70ae2e3 (0xc5a7 bytes) 2024-09-12T07:20:22.743 INFO:teuthology.orchestra.run.smithi046.stdout:2636 punch from 0x71b4b5b to 0x71bc5f7, (0x7a9c bytes) 2024-09-12T07:20:22.743 INFO:teuthology.orchestra.run.smithi046.stdout:2637 read 0x8bd6732 thru 0x8bdf0b9 (0x8988 bytes) 2024-09-12T07:20:22.743 INFO:teuthology.orchestra.run.smithi046.stdout:2638 read 0x8824eee thru 0x8824f4b (0x5e bytes) 2024-09-12T07:20:22.743 INFO:teuthology.orchestra.run.smithi046.stdout:2639 punch from 0x2f05b3c to 0x2f0ff48, (0xa40c bytes) 2024-09-12T07:20:22.743 INFO:teuthology.orchestra.run.smithi046.stdout:2643 write 0x396759b thru 0x3967bb3 (0x619 bytes) 2024-09-12T07:20:22.744 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:22.744 INFO:teuthology.orchestra.run.smithi046.stdout:2645 read 0x356c29a thru 0x3571273 (0x4fda bytes) 2024-09-12T07:20:22.744 INFO:teuthology.orchestra.run.smithi046.stdout:2648 trunc from 0x8f6fd30 to 0x8ee826c 2024-09-12T07:20:22.751 INFO:teuthology.orchestra.run.smithi046.stdout:2650 trunc from 0x8ee826c to 0x9d50a13 2024-09-12T07:20:22.758 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:22.759+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:22.766 INFO:teuthology.orchestra.run.smithi046.stdout:2653 write 0xd1ab21d thru 0xd1af5db (0x43bf bytes) 2024-09-12T07:20:22.769 INFO:teuthology.orchestra.run.smithi046.stdout:2654 read 0x14bedf0 thru 0x14c9fd7 (0xb1e8 bytes) 2024-09-12T07:20:22.770 INFO:teuthology.orchestra.run.smithi046.stdout:2655 clone 21 order 18 su 4096 sc 10 2024-09-12T07:20:22.786 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:22.786 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:22.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:22.786 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:894 3.2 3:468e8e09:::rbd_data.116011b0bca6.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.684295011000003, 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015762987999999999, 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 894 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.787 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102913+0000", 2024-09-12T07:20:22.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004e-06 2024-09-12T07:20:22.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102918+0000", 2024-09-12T07:20:22.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6260000000000003e-06 2024-09-12T07:20:22.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102929+0000", 2024-09-12T07:20:22.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1382e-05 2024-09-12T07:20:22.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102934+0000", 2024-09-12T07:20:22.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:20:22.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102988+0000", 2024-09-12T07:20:22.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3683000000000001e-05 2024-09-12T07:20:22.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.103072+0000", 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4135000000000007e-05 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.105687+0000", 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026149580000000001 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:22.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118570+0000", 2024-09-12T07:20:22.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012883034 2024-09-12T07:20:22.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:22.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118585+0000", 2024-09-12T07:20:22.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4797000000000001e-05 2024-09-12T07:20:22.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:22.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118622+0000", 2024-09-12T07:20:22.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7163000000000003e-05 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118675+0000", 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3236000000000003e-05 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:88696 3.c e53/46 3:35e89a4f:::rbd_data.11607a56846.0000000000000014:head v 53'62, mlcod=53'60)", 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:13.501634+0000", 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2855724649999996, 2024-09-12T07:20:22.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013681594, 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.501634+0000", 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.501634+0000", 2024-09-12T07:20:22.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.501636+0000", 2024-09-12T07:20:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.668e-06 2024-09-12T07:20:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.508127+0000", 2024-09-12T07:20:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064905789999999998 2024-09-12T07:20:22.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.508152+0000", 2024-09-12T07:20:22.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5208e-05 2024-09-12T07:20:22.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.508165+0000", 2024-09-12T07:20:22.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2677000000000001e-05 2024-09-12T07:20:22.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.508243+0000", 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8857999999999996e-05 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.508274+0000", 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0954999999999998e-05 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.515249+0000", 2024-09-12T07:20:22.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069747919999999996 2024-09-12T07:20:22.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.515316+0000", 2024-09-12T07:20:22.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6857e-05 2024-09-12T07:20:22.798 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.798 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.798 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.798 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:88838 3.2 3:407ead74:::rbd_data.11607a56846.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 2220620~4647] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:13.589368+0000", 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1978387710000007, 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012610503, 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 88838 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.589368+0000", 2024-09-12T07:20:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.589368+0000", 2024-09-12T07:20:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.589369+0000", 2024-09-12T07:20:22.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.268e-06 2024-09-12T07:20:22.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.590751+0000", 2024-09-12T07:20:22.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001381125 2024-09-12T07:20:22.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.590780+0000", 2024-09-12T07:20:22.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9173e-05 2024-09-12T07:20:22.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.590794+0000", 2024-09-12T07:20:22.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4062e-05 2024-09-12T07:20:22.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.590878+0000", 2024-09-12T07:20:22.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3963999999999998e-05 2024-09-12T07:20:22.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.591062+0000", 2024-09-12T07:20:22.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018438300000000001 2024-09-12T07:20:22.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:20:22.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.591511+0000", 2024-09-12T07:20:22.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044836300000000002 2024-09-12T07:20:22.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:22.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.599382+0000", 2024-09-12T07:20:22.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078715820000000002 2024-09-12T07:20:22.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:22.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.601849+0000", 2024-09-12T07:20:22.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024668369999999999 2024-09-12T07:20:22.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:22.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.601914+0000", 2024-09-12T07:20:22.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5306000000000001e-05 2024-09-12T07:20:22.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.601979+0000", 2024-09-12T07:20:22.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.444e-05 2024-09-12T07:20:22.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.810 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.811 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:88880 3.18 3:1b2fb8d6:::rbd_data.11607a56846.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 2129920~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-12T07:20:22.811 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:13.609971+0000", 2024-09-12T07:20:22.811 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1772355409999999, 2024-09-12T07:20:22.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011637115, 2024-09-12T07:20:22.812 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:22.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.813 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.813 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.813 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 88880 2024-09-12T07:20:22.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.813 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.609971+0000", 2024-09-12T07:20:22.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.609971+0000", 2024-09-12T07:20:22.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.609972+0000", 2024-09-12T07:20:22.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.046e-06 2024-09-12T07:20:22.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.611254+0000", 2024-09-12T07:20:22.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001281882 2024-09-12T07:20:22.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.611275+0000", 2024-09-12T07:20:22.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0791000000000001e-05 2024-09-12T07:20:22.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.611284+0000", 2024-09-12T07:20:22.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0899999999999994e-06 2024-09-12T07:20:22.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.611366+0000", 2024-09-12T07:20:22.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1933000000000003e-05 2024-09-12T07:20:22.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.611539+0000", 2024-09-12T07:20:22.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172771 2024-09-12T07:20:22.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:20:22.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.611866+0000", 2024-09-12T07:20:22.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032747000000000001 2024-09-12T07:20:22.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:22.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.620730+0000", 2024-09-12T07:20:22.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088641009999999992 2024-09-12T07:20:22.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:22.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.621487+0000", 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00075658099999999999 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.621557+0000", 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9521000000000001e-05 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.621609+0000", 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1928999999999997e-05 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:90123 3.2 3:407ead74:::rbd_data.11607a56846.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:14.259915+0000", 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.5272915359999999, 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010349486999999999, 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.822 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 90123 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259915+0000", 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259915+0000", 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259916+0000", 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0279999999999999e-06 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259921+0000", 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8930000000000004e-06 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259934+0000", 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2381000000000001e-05 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259949+0000", 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5729000000000001e-05 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259994+0000", 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4860000000000001e-05 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.260047+0000", 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2468e-05 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.261523+0000", 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014767300000000001 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.270189+0000", 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086655619999999999 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.270200+0000", 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1345999999999999e-05 2024-09-12T07:20:22.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.270233+0000", 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2141000000000003e-05 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.270265+0000", 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2348999999999998e-05 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:90226 3.8 3:15b350ab:::rbd_data.11607a56846.000000000000001c:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:14.355994+0000", 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.4312125929999997, 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012285943000000001, 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 90226 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.355994+0000", 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.355994+0000", 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.355996+0000", 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.367e-06 2024-09-12T07:20:22.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.356002+0000", 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0419999999999999e-06 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.356015+0000", 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3543000000000001e-05 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.356023+0000", 2024-09-12T07:20:22.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6450000000000005e-06 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.356075+0000", 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460999999999998e-05 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.356147+0000", 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1612999999999994e-05 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-12T07:20:22.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.358786+0000", 2024-09-12T07:20:22.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002639045 2024-09-12T07:20:22.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:22.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.368142+0000", 2024-09-12T07:20:22.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093560329999999997 2024-09-12T07:20:22.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:22.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.368162+0000", 2024-09-12T07:20:22.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0343000000000002e-05 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.368225+0000", 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2069999999999994e-05 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.368280+0000", 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5781e-05 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97197 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3457024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.688930+0000", 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0982766169999998, 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093541869999999999, 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97197 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.834 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688930+0000", 2024-09-12T07:20:22.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688930+0000", 2024-09-12T07:20:22.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688931+0000", 2024-09-12T07:20:22.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0699999999999999e-07 2024-09-12T07:20:22.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688934+0000", 2024-09-12T07:20:22.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8700000000000002e-06 2024-09-12T07:20:22.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688939+0000", 2024-09-12T07:20:22.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8609999999999997e-06 2024-09-12T07:20:22.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688948+0000", 2024-09-12T07:20:22.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2019999999999993e-06 2024-09-12T07:20:22.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698105+0000", 2024-09-12T07:20:22.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.009156951 2024-09-12T07:20:22.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698196+0000", 2024-09-12T07:20:22.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0845000000000002e-05 2024-09-12T07:20:22.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698285+0000", 2024-09-12T07:20:22.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8251000000000006e-05 2024-09-12T07:20:22.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.840 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.840 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97198 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:22.841 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.688955+0000", 2024-09-12T07:20:22.841 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0982516459999996, 2024-09-12T07:20:22.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094347690000000008, 2024-09-12T07:20:22.841 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.841 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97198 2024-09-12T07:20:22.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.842 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688955+0000", 2024-09-12T07:20:22.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688955+0000", 2024-09-12T07:20:22.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688956+0000", 2024-09-12T07:20:22.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9700000000000003e-07 2024-09-12T07:20:22.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688957+0000", 2024-09-12T07:20:22.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4720000000000001e-06 2024-09-12T07:20:22.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688961+0000", 2024-09-12T07:20:22.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8399999999999997e-06 2024-09-12T07:20:22.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688964+0000", 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7e-06 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698310+0000", 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093460009999999996 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698350+0000", 2024-09-12T07:20:22.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0630000000000002e-05 2024-09-12T07:20:22.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698390+0000", 2024-09-12T07:20:22.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9829000000000001e-05 2024-09-12T07:20:22.848 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.848 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97199 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.688967+0000", 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0982399049999998, 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096352060000000003, 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.849 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97199 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688967+0000", 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688967+0000", 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688967+0000", 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1e-07 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688975+0000", 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1149999999999994e-06 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688981+0000", 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2429999999999996e-06 2024-09-12T07:20:22.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688983+0000", 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5749999999999999e-06 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698425+0000", 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094418430000000001 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698516+0000", 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0897999999999996e-05 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698602+0000", 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6322000000000005e-05 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97200 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:22.853 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.688987+0000", 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0982201949999997, 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0097196250000000008, 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97200 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688987+0000", 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688987+0000", 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688987+0000", 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.36e-07 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688988+0000", 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2300000000000001e-06 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688993+0000", 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8420000000000004e-06 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688996+0000", 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4509999999999999e-06 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698627+0000", 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096317650000000005 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698668+0000", 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0819999999999999e-05 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698706+0000", 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8281000000000002e-05 2024-09-12T07:20:22.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97201 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.688999+0000", 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0982081849999998, 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099215780000000003, 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97201 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688999+0000", 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688999+0000", 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688999+0000", 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.16e-07 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689000+0000", 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0240000000000001e-06 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689003+0000", 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0769999999999999e-06 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689006+0000", 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.491e-06 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698740+0000", 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0097341709999999998 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698831+0000", 2024-09-12T07:20:22.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0821000000000003e-05 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698920+0000", 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9777999999999993e-05 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97202 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.689009+0000", 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0981982410000004, 2024-09-12T07:20:22.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010021258, 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97202 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689009+0000", 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689009+0000", 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689009+0000", 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4400000000000001e-07 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689010+0000", 2024-09-12T07:20:22.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0759999999999999e-06 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689013+0000", 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9409999999999999e-06 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689016+0000", 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.253e-06 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698950+0000", 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099334829999999999 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698991+0000", 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1619000000000003e-05 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699030+0000", 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8642e-05 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97203 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.689021+0000", 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0981860880000003, 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010221196, 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97203 2024-09-12T07:20:22.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689021+0000", 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689021+0000", 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689021+0000", 2024-09-12T07:20:22.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5699999999999998e-07 2024-09-12T07:20:22.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689022+0000", 2024-09-12T07:20:22.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0550000000000001e-06 2024-09-12T07:20:22.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689030+0000", 2024-09-12T07:20:22.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5449999999999998e-06 2024-09-12T07:20:22.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689034+0000", 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8430000000000003e-06 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699067+0000", 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010033432 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699158+0000", 2024-09-12T07:20:22.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0420000000000005e-05 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699242+0000", 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4543999999999996e-05 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97204 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.689037+0000", 2024-09-12T07:20:22.871 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0981696100000002, 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010307439, 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97204 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689037+0000", 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689037+0000", 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689038+0000", 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.34e-07 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689039+0000", 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1939999999999999e-06 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689042+0000", 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2559999999999998e-06 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689045+0000", 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4760000000000001e-06 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699267+0000", 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010222825 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699306+0000", 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8463000000000001e-05 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699345+0000", 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8890999999999997e-05 2024-09-12T07:20:22.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97205 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 4145152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.689048+0000", 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0981585389999999, 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010502008, 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97205 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689048+0000", 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689048+0000", 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689049+0000", 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.23e-07 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689050+0000", 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689053+0000", 2024-09-12T07:20:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0089999999999999e-06 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689055+0000", 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4320000000000002e-06 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699371+0000", 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010316201000000001 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699465+0000", 2024-09-12T07:20:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3599999999999998e-05 2024-09-12T07:20:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699550+0000", 2024-09-12T07:20:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5477999999999997e-05 2024-09-12T07:20:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:100373 3.8 3:16d49e60:::rbd_data.1160d60c02f0.0000000000000005:15 [read 1376256~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:21.479532+0000", 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.30767473, 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095017790000000001, 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100373 2024-09-12T07:20:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479532+0000", 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479532+0000", 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479533+0000", 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3000000000000001e-07 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479535+0000", 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479543+0000", 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1310000000000006e-06 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479550+0000", 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1409999999999999e-06 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.488839+0000", 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092886740000000002 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.488924+0000", 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5410999999999999e-05 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489034+0000", 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010980900000000001 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:100374 3.8 3:16d49e60:::rbd_data.1160d60c02f0.0000000000000005:15 [read 1605632~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:21.479558+0000", 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3076487210000001, 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0097197859999999994, 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100374 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479558+0000", 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479558+0000", 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479559+0000", 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2500000000000001e-07 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479561+0000", 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2919999999999998e-06 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479569+0000", 2024-09-12T07:20:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3250000000000008e-06 2024-09-12T07:20:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479576+0000", 2024-09-12T07:20:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9480000000000002e-06 2024-09-12T07:20:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489081+0000", 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095043899999999997 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489167+0000", 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6790000000000001e-05 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489278+0000", 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110616 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:100375 3.8 3:16d49e60:::rbd_data.1160d60c02f0.0000000000000005:15 [read 1835008~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:21.479586+0000", 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3076213210000001, 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099179139999999999, 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100375 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479586+0000", 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479586+0000", 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479586+0000", 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479588+0000", 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.024e-06 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479596+0000", 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1680000000000004e-06 2024-09-12T07:20:22.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479603+0000", 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6490000000000002e-06 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489321+0000", 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0097178560000000004 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489400+0000", 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9059000000000003e-05 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489504+0000", 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103705 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:100376 3.8 3:16d49e60:::rbd_data.1160d60c02f0.0000000000000005:15 [read 2064384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:21.479611+0000", 2024-09-12T07:20:22.894 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.3075957439999999, 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010113561, 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100376 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479611+0000", 2024-09-12T07:20:22.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479611+0000", 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479612+0000", 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5499999999999998e-07 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479614+0000", 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.934e-06 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479622+0000", 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2179999999999999e-06 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479628+0000", 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5830000000000003e-06 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489534+0000", 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099051980000000005 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489615+0000", 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1787999999999997e-05 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489725+0000", 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109385 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.898 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.899 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:100438 3.2 3:46b60448:::rbd_data.11606e757b33.000000000000005e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-12T07:20:22.899 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:21.550725+0000", 2024-09-12T07:20:22.899 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.2364817910000001, 2024-09-12T07:20:22.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010300614, 2024-09-12T07:20:22.899 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:22.900 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:22.900 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:22.900 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:22.900 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:22.900 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:22.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100438 2024-09-12T07:20:22.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.900 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:22.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:22.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550725+0000", 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550725+0000", 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550726+0000", 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550731+0000", 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9119999999999997e-06 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550742+0000", 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1233000000000001e-05 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550747+0000", 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0760000000000002e-06 2024-09-12T07:20:22.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550801+0000", 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4012999999999999e-05 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550882+0000", 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0953999999999998e-05 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.552616+0000", 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017335079999999999 2024-09-12T07:20:22.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.560932+0000", 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083159500000000008 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.560968+0000", 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6254000000000002e-05 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.561000+0000", 2024-09-12T07:20:22.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1506e-05 2024-09-12T07:20:22.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:22.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:22.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:22.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.561026+0000", 2024-09-12T07:20:22.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6250000000000001e-05 2024-09-12T07:20:22.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.905 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:22.905 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:22.905 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:22.907 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:22.907 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:22.908+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:23.008 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:23.010+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:23.015 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:23.015 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:23.015 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:23.015 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:23.023 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:23.109 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:23.111+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:23.130 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:23.130 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:23.130 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:23.130 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:23.130 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:23.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:23.210 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:23.211+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:23.247 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:23.247 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:23.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:23.247 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:23.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.247 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:20:23.247 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 25.634700514999999, 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:20:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:20:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:20:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:20:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:20:23.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 20.499240259, 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:20:23.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:20:23.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 20.495013138000001, 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.260 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.767262483, 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:20:23.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67907 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'26, mlcod=43'22)", 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.500559+0000", 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.747451836, 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014503131000000001, 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.265 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.500559+0000", 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.500559+0000", 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.500561+0000", 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.497e-06 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.500570+0000", 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2029999999999998e-06 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.500589+0000", 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8476000000000001e-05 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.500598+0000", 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6900000000000004e-06 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.500664+0000", 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5820000000000003e-05 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.500680+0000", 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5709000000000002e-05 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.515023+0000", 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014343333999999999 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.515063+0000", 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:20:23.269 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.678285893999998, 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68853 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'43, mlcod=43'39)", 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.667293307000001, 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017793166999999999, 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:23.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580719+0000", 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4720000000000001e-06 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580732+0000", 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2926e-05 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580746+0000", 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3291e-05 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580755+0000", 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6709999999999994e-06 2024-09-12T07:20:23.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583766+0000", 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003010432 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583794+0000", 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.811e-05 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598408+0000", 2024-09-12T07:20:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014614472 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598511+0000", 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102793 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:23.278 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.646734563999999, 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:20:23.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.643534028000001, 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:20:23.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:20:23.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:20:23.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "pg_update_log_missing(3.e epoch 43/24 rep_tid 516 entries 43'92 (0'0) error 3:766c40a0:::rbd_data.1160f022dd61.0000000000000014:head by client.4448.0:68858 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 41'90)", 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607635+0000", 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.640375955, 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011395885999999999, 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607635+0000", 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607635+0000", 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607636+0000", 2024-09-12T07:20:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7699999999999992e-07 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607646+0000", 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7119999999999995e-06 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607681+0000", 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5135999999999997e-05 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607695+0000", 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4171999999999999e-05 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607793+0000", 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8047999999999994e-05 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.619031+0000", 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011237841 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.64028828, 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:23.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:20:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.638046449000001, 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:20:23.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.299 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69441 3.1b e43/24 3:df314e83:::rbd_data.1160f022dd61.0000000000000011:head v 43'44, mlcod=43'41)", 2024-09-12T07:20:23.300 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.681840+0000", 2024-09-12T07:20:23.300 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.566171744999998, 2024-09-12T07:20:23.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014496068000000001, 2024-09-12T07:20:23.300 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.300 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.300 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.300 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.681840+0000", 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.681840+0000", 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.681841+0000", 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.81e-06 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.688493+0000", 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0066520599999999996 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.688535+0000", 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1779000000000001e-05 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.688545+0000", 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4839999999999993e-06 2024-09-12T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.688615+0000", 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0314999999999993e-05 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.688638+0000", 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3085e-05 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.696284+0000", 2024-09-12T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0076457560000000001 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.696336+0000", 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1779e-05 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69559 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'29, mlcod=41'27)", 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.563715401, 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018119258999999999, 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684297+0000", 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.192e-06 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691509+0000", 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072122239999999997 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691524+0000", 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4562999999999999e-05 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691533+0000", 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1069999999999994e-06 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691598+0000", 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4666999999999996e-05 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691620+0000", 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2092000000000001e-05 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702352+0000", 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010731785000000001 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702415+0000", 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3628999999999997e-05 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.308 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69560 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'31, mlcod=41'27)", 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.556149277999999, 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016047555000000002, 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691863+0000", 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0299999999999999e-07 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691897+0000", 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4765999999999999e-05 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691926+0000", 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8413000000000001e-05 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691934+0000", 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3839999999999997e-06 2024-09-12T07:20:23.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694739+0000", 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028043640000000002 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694773+0000", 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4514000000000002e-05 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707854+0000", 2024-09-12T07:20:23.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013080591000000001 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707910+0000", 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.592e-05 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "pg_update_log_missing(3.1a epoch 43/24 rep_tid 362 entries 43'32 (0'0) error 3:5ec4a676:::rbd_data.1160f022dd61.0000000000000019:head by client.4448.0:69639 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 41'27)", 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.555621563999999, 2024-09-12T07:20:23.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015537618, 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692497+0000", 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107154 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692519+0000", 2024-09-12T07:20:23.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1043000000000002e-05 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692542+0000", 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3866000000000002e-05 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702496+0000", 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0099539169999999996 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707927+0000", 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005431038 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.546939245000001, 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.317 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:20:23.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.543111156999998, 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:20:23.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:20:23.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89144 3.a e53/24 3:54996542:::rbd_data.11607a56846.0000000000000001:head v 53'83, mlcod=53'81)", 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.554954403, 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018298937000000001, 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.326 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693059+0000", 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.565e-06 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696238+0000", 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0031783269999999999 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696265+0000", 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6982e-05 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696279+0000", 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4438e-05 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696366+0000", 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6978000000000006e-05 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696397+0000", 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0406999999999999e-05 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711297+0000", 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014900689999999999 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711356+0000", 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.855e-05 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89145 3.13 e53/24 3:ca7c3842:::rbd_data.11607a56846.0000000000000002:head v 53'202, mlcod=53'199)", 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.5539230600000007, 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019835603, 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694091+0000", 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4990000000000001e-06 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697616+0000", 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035256749999999998 2024-09-12T07:20:23.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697635+0000", 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8593999999999998e-05 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697646+0000", 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1436e-05 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:23.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697717+0000", 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0679000000000005e-05 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697748+0000", 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0723999999999999e-05 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713886+0000", 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016138550000000002 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:23.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:23.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713924+0000", 2024-09-12T07:20:23.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7446e-05 2024-09-12T07:20:23.335 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.335 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.335 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.335 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:23.335 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:23.335 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:20:23.337 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:23.338 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone20 from 0xd1af5dc (overlap 0x39dbc94) to 0x2880401 2024-09-12T07:20:23.338 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:23.338+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:23.437 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:23.439+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:23.443 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-12T07:20:23.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:20:23.503 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:23.503 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:23.504 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:23.504 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:23.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:23.519 INFO:teuthology.orchestra.run.smithi046.stdout:2656 read 0x492609a thru 0x492c23b (0x61a2 bytes) 2024-09-12T07:20:23.520 INFO:teuthology.orchestra.run.smithi046.stdout:2657 read 0x434fe7e thru 0x435c91d (0xcaa0 bytes) 2024-09-12T07:20:23.521 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:23.532 INFO:teuthology.orchestra.run.smithi046.stdout:2659 trunc from 0xd1af5dc to 0x2b3007a 2024-09-12T07:20:23.557 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:23.559+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:23.597 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:23.597 INFO:teuthology.orchestra.run.smithi046.stdout:2661 punch from 0x1416888 to 0x141f6e3, (0x8e5b bytes) 2024-09-12T07:20:23.603 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:23.607 INFO:teuthology.orchestra.run.smithi046.stdout:2662 write 0x6c6576f thru 0x6c65c8e (0x520 bytes) 2024-09-12T07:20:23.610 INFO:teuthology.orchestra.run.smithi046.stdout:2663 read 0x3dcc3d7 thru 0x3dd7a8a (0xb6b4 bytes) 2024-09-12T07:20:23.611 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:23.611 INFO:teuthology.orchestra.run.smithi046.stdout:2665 read 0x617125f thru 0x617fcbf (0xea61 bytes) 2024-09-12T07:20:23.611 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:23.611 INFO:teuthology.orchestra.run.smithi046.stdout:2667 read 0x128753f thru 0x1287ef2 (0x9b4 bytes) 2024-09-12T07:20:23.611 INFO:teuthology.orchestra.run.smithi046.stdout:2668 read 0xfd9a4f thru 0xfdd2ab (0x385d bytes) 2024-09-12T07:20:23.620 INFO:teuthology.orchestra.run.smithi046.stdout:2669 write 0x97e0159 thru 0x97e26f9 (0x25a1 bytes) 2024-09-12T07:20:23.622 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:23.623 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:23.623 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:23.623 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:23.623 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:23.623 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:23.623 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:23.632 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:23.637 INFO:teuthology.orchestra.run.smithi046.stdout:2671 write 0xc39d0bd thru 0xc3a6d98 (0x9cdc bytes) 2024-09-12T07:20:23.639 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:23.641+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:23.647 INFO:teuthology.orchestra.run.smithi046.stdout:2672 read 0x333d156 thru 0x334c40c (0xf2b7 bytes) 2024-09-12T07:20:23.647 INFO:teuthology.orchestra.run.smithi046.stdout:2674 read 0x3bdc36d thru 0x3be369e (0x7332 bytes) 2024-09-12T07:20:23.647 INFO:teuthology.orchestra.run.smithi046.stdout:2675 write 0x8d61823 thru 0x8d62e54 (0x1632 bytes) 2024-09-12T07:20:23.649 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:23.649 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:23.651 INFO:teuthology.orchestra.run.smithi046.stdout:2676 write 0x5c1c6a9 thru 0x5c210ef (0x4a47 bytes) 2024-09-12T07:20:23.664 INFO:teuthology.orchestra.run.smithi046.stdout:2678 writesame 0xc6b6f4f thru 0xc6b772b data_size 0xb(0x7dd bytes) 2024-09-12T07:20:23.664 INFO:teuthology.orchestra.run.smithi046.stdout:2679 write 0x990a6af thru 0x99199c9 (0xf31b bytes) 2024-09-12T07:20:23.671 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:23.676 INFO:teuthology.orchestra.run.smithi046.stdout:2680 write 0x9104359 thru 0x9108efa (0x4ba2 bytes) 2024-09-12T07:20:23.682 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:23.683 INFO:teuthology.orchestra.run.smithi046.stdout:2683 clone 22 order 24 su 262144 sc 7 2024-09-12T07:20:23.692 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:23.709 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:23.709 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:23.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:23.726 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:23.740 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:23.741+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:23.744 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:23.744 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:23.782 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:23.782 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:23.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:23.841 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:23.843+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:23.898 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:23.898 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:23.906 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:23.942 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:23.943+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:24.045 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:24.046+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:24.053 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:24.053 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:24.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:24.145 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:24.147+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:24.167 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:24.168 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:24.176 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:24.246 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:24.248+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:24.285 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:24.285 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:24.295 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:24.358 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone21 from 0xc6b772c (overlap 0x2b3007a) to 0xcf4cf1 2024-09-12T07:20:24.378 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:24.380+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:24.448 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:24.450+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:24.459 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:24.459 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:24.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:24.552 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:24.554+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:24.616 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:24.616 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:24.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:24.668 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:24.670+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:24.732 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:24.733 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:24.741 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:24.753 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:24.754+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:24.854 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:24.856+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:24.943 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:24.943 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:24.953 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:24.955 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:24.956+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:25.056 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:25.058+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:25.099 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:25.099 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:25.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:25.158 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:25.159+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:25.253 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:25.253 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:25.258 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:25.260+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:25.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:25.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:20:25.366 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-12T07:20:25.366 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:25.367+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:25.449 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:25.450 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:25.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:25.461 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:25.461+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:25.482 INFO:teuthology.orchestra.run.smithi046.stdout:2684 write 0x75d1772 thru 0x75e110e (0xf99d bytes) 2024-09-12T07:20:25.493 INFO:teuthology.orchestra.run.smithi046.stdout:2685 read 0x242f37 thru 0x25040c (0xd4d6 bytes) 2024-09-12T07:20:25.496 INFO:teuthology.orchestra.run.smithi046.stdout:2686 write 0x535be96 thru 0x53640fa (0x8265 bytes) 2024-09-12T07:20:25.561 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:25.562+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:25.622 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:25.622 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:25.626 INFO:teuthology.orchestra.run.smithi046.stdout:2687 write 0xd5e1885 thru 0xd5ea370 (0x8aec bytes) 2024-09-12T07:20:25.632 INFO:teuthology.orchestra.run.smithi046.stdout:2688 writesame 0x7edc319 thru 0x7ee78f9 data_size 0x1(0xb5e1 bytes) 2024-09-12T07:20:25.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:25.662 INFO:teuthology.orchestra.run.smithi046.stdout:2689 write 0x21de953 thru 0x21e688b (0x7f39 bytes) 2024-09-12T07:20:25.663 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:25.664+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:25.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:25.763 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:25.764+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:25.769 INFO:teuthology.orchestra.run.smithi046.stdout:2690 write 0x52ecef9 thru 0x52f43e3 (0x74eb bytes) 2024-09-12T07:20:25.775 INFO:teuthology.orchestra.run.smithi046.stdout:2691 trunc from 0xd5ea371 to 0xa0ccabc 2024-09-12T07:20:25.804 INFO:teuthology.orchestra.run.smithi046.stdout:2692 write 0xc7ce583 thru 0xc7d12f1 (0x2d6f bytes) 2024-09-12T07:20:25.805 INFO:teuthology.orchestra.run.smithi046.stdout:2693 clone 23 order 20 su 8192 sc 12 2024-09-12T07:20:25.820 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:25.820 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:25.830 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:25.870 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:25.872+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:25.900 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:25.917 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:25.917 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:25.934 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:25.951 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:25.951 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:25.964 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:25.966+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:25.968 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:25.978 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:25.978 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:25.978 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:25.978 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:25.978 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:25.985 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:25.987 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:26.001 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:26.018 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:26.018 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:26.066 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:26.067+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:26.110 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77440 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13156352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.866107193000001, 2024-09-12T07:20:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051399724000000001, 2024-09-12T07:20:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77440 2024-09-12T07:20:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246267+0000", 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082256199999999999 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246271+0000", 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6700000000000002e-06 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246283+0000", 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.207e-05 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246291+0000", 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296645+0000", 2024-09-12T07:20:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050353872000000001 2024-09-12T07:20:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296729+0000", 2024-09-12T07:20:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4215000000000006e-05 2024-09-12T07:20:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296844+0000", 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114626 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77446 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.865108677999999, 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051543643, 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77446 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246579+0000", 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013651399999999999 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246583+0000", 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1509999999999997e-06 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246593+0000", 2024-09-12T07:20:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5670000000000001e-06 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246599+0000", 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5189999999999998e-06 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297826+0000", 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051226381000000001 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297903+0000", 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6710000000000002e-05 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297986+0000", 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3801000000000006e-05 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77447 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 5783552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.864941064, 2024-09-12T07:20:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051512637, 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77447 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246812+0000", 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000201506 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246817+0000", 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246828+0000", 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1693e-05 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246836+0000", 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4429999999999997e-06 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298021+0000", 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051184940999999998 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298067+0000", 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6292000000000003e-05 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298123+0000", 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5828999999999997e-05 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77448 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.864701325999999, 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051506024999999997, 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77448 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246892+0000", 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1854999999999998e-05 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246895+0000", 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3129999999999999e-06 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246904+0000", 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6179999999999995e-06 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246911+0000", 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7939999999999997e-06 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298164+0000", 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051253559999999997 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298249+0000", 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4549000000000007e-05 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298356+0000", 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107336 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77449 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.864630129999998, 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051582830000000003, 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77449 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247137+0000", 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021599499999999999 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247142+0000", 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247154+0000", 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1878e-05 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247168+0000", 2024-09-12T07:20:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4114000000000001e-05 2024-09-12T07:20:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298391+0000", 2024-09-12T07:20:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051223249999999998 2024-09-12T07:20:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298446+0000", 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4366000000000001e-05 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298504+0000", 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8273000000000003e-05 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77450 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12681216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.864370560000001, 2024-09-12T07:20:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051562375000000001, 2024-09-12T07:20:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77450 2024-09-12T07:20:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2799999999999996e-07 2024-09-12T07:20:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247184+0000", 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247192+0000", 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5920000000000003e-06 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247197+0000", 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6720000000000001e-06 2024-09-12T07:20:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298550+0000", 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051352082 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298641+0000", 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1236000000000005e-05 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298743+0000", 2024-09-12T07:20:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102428 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77451 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7061504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.864346445999999, 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051813773, 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77451 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247335+0000", 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013012599999999999 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247340+0000", 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.544e-06 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247351+0000", 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.111e-05 2024-09-12T07:20:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247358+0000", 2024-09-12T07:20:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9859999999999997e-06 2024-09-12T07:20:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298909+0000", 2024-09-12T07:20:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051551781999999997 2024-09-12T07:20:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298960+0000", 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0594999999999999e-05 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299019+0000", 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8629999999999999e-05 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77452 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.864181666, 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051892134999999999, 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77452 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247535+0000", 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165508 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247552+0000", 2024-09-12T07:20:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6514000000000001e-05 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247565+0000", 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3372e-05 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247577+0000", 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299064+0000", 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051486697999999997 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299150+0000", 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6549000000000001e-05 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299262+0000", 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011135 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77453 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.863960944999999, 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051901623000000001, 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77453 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247710+0000", 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011951 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247713+0000", 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4539999999999999e-06 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247720+0000", 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0339999999999999e-06 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247730+0000", 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0030000000000001e-05 2024-09-12T07:20:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299297+0000", 2024-09-12T07:20:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051566446000000002 2024-09-12T07:20:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299382+0000", 2024-09-12T07:20:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4963000000000006e-05 2024-09-12T07:20:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299492+0000", 2024-09-12T07:20:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011018600000000001 2024-09-12T07:20:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77454 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7618560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.863812398, 2024-09-12T07:20:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051921851999999998, 2024-09-12T07:20:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77454 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247900+0000", 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016088699999999999 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247904+0000", 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6950000000000002e-06 2024-09-12T07:20:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247912+0000", 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7129999999999994e-06 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247921+0000", 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0299999999999999e-06 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299526+0000", 2024-09-12T07:20:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051604744000000001 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299601+0000", 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5129000000000005e-05 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299661+0000", 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9654000000000002e-05 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77455 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12746752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.86361853, 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051971254000000001, 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77455 2024-09-12T07:20:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248181+0000", 2024-09-12T07:20:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024790000000000001 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248185+0000", 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5079999999999999e-06 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248197+0000", 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.132e-05 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248203+0000", 2024-09-12T07:20:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6320000000000002e-06 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299705+0000", 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051501342999999998 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299799+0000", 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4340999999999997e-05 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299904+0000", 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010521 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77456 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7979008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.863336305000001, 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051860577999999997, 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77456 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248545+0000", 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033009700000000001 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248550+0000", 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248560+0000", 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248568+0000", 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4939999999999997e-06 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299973+0000", 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051405470000000002 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300019+0000", 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5448999999999998e-05 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300076+0000", 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7073000000000001e-05 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77457 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13172736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.862969976999999, 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051737914000000003, 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77457 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248582+0000", 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248584+0000", 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248593+0000", 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248609+0000", 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5664000000000001e-05 2024-09-12T07:20:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300117+0000", 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051507605999999997 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300212+0000", 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4919000000000004e-05 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300319+0000", 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010776399999999999 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77458 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.862927558999999, 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051863578, 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77458 2024-09-12T07:20:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:20:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248627+0000", 2024-09-12T07:20:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1389999999999999e-06 2024-09-12T07:20:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248635+0000", 2024-09-12T07:20:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4019999999999996e-06 2024-09-12T07:20:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248641+0000", 2024-09-12T07:20:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1689999999999997e-06 2024-09-12T07:20:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300359+0000", 2024-09-12T07:20:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051717889000000003 2024-09-12T07:20:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300418+0000", 2024-09-12T07:20:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8689e-05 2024-09-12T07:20:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300487+0000", 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9756000000000001e-05 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77459 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13238272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.862901866000001, 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052076504000000003, 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77459 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250611+0000", 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019617879999999999 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250618+0000", 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1829999999999997e-06 2024-09-12T07:20:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250629+0000", 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0438000000000001e-05 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250648+0000", 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9355e-05 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300528+0000", 2024-09-12T07:20:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049879252999999998 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300619+0000", 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1910000000000006e-05 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300726+0000", 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010657700000000001 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77467 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.860696047000001, 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05160754, 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77467 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250856+0000", 2024-09-12T07:20:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250858+0000", 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9180000000000001e-06 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250864+0000", 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.066e-06 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250869+0000", 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302283+0000", 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051414591000000003 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302378+0000", 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4283000000000005e-05 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302463+0000", 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5249999999999999e-05 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77468 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14909440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.860676474000002, 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051742162000000001, 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77468 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250877+0000", 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.815e-06 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250883+0000", 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250888+0000", 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302496+0000", 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051607394000000001 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302553+0000", 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6971000000000001e-05 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302617+0000", 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4418999999999994e-05 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77469 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14974976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.860656294999998, 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965694, 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77469 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250897+0000", 2024-09-12T07:20:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9130000000000001e-06 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250903+0000", 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0129999999999999e-06 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250908+0000", 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9679999999999997e-06 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302661+0000", 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051753066 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302755+0000", 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3376999999999997e-05 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302861+0000", 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105924 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77470 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12828672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.860636618000001, 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052098589000000001, 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77470 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250917+0000", 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8190000000000001e-06 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250923+0000", 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250928+0000", 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9420000000000003e-06 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302894+0000", 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965863000000001 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302955+0000", 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1235999999999994e-05 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303013+0000", 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8167000000000001e-05 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77471 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12894208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.860617072, 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052326263999999997, 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77471 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256961+0000", 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060271209999999999 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256972+0000", 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0755e-05 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256986+0000", 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845e-05 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256996+0000", 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303060+0000", 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.046063661999999998 2024-09-12T07:20:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303149+0000", 2024-09-12T07:20:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8949999999999994e-05 2024-09-12T07:20:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303261+0000", 2024-09-12T07:20:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011195 2024-09-12T07:20:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.201 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:26.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:26.203 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:26.204+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:26.268 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:20:26.268 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-12T07:20:26.274 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:20:26.303 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:26.305+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:26.359 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone22 from 0xc7d12f2 (overlap 0xa0ccabc) to 0x607acd7 2024-09-12T07:20:26.366 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:26.366 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:26.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:26.404 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:26.405+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:26.505 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:26.506+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:26.573 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:26.573 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:26.573 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:26.573 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:26.573 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:26.587 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:26.605 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:26.607+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:26.707 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:26.708+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:26.744 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:894 3.2 3:468e8e09:::rbd_data.116011b0bca6.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T07:20:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.641902373000001, 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015762987999999999, 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 894 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:20:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102912+0000", 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102913+0000", 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004e-06 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102918+0000", 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6260000000000003e-06 2024-09-12T07:20:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102929+0000", 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1382e-05 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102934+0000", 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.102988+0000", 2024-09-12T07:20:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3683000000000001e-05 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.103072+0000", 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4135000000000007e-05 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.105687+0000", 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026149580000000001 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118570+0000", 2024-09-12T07:20:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012883034 2024-09-12T07:20:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118585+0000", 2024-09-12T07:20:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4797000000000001e-05 2024-09-12T07:20:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118622+0000", 2024-09-12T07:20:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7163000000000003e-05 2024-09-12T07:20:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:19:48.118675+0000", 2024-09-12T07:20:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3236000000000003e-05 2024-09-12T07:20:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4448.0:88696 3.c e53/46 3:35e89a4f:::rbd_data.11607a56846.0000000000000014:head v 53'62, mlcod=53'60)", 2024-09-12T07:20:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:13.501634+0000", 2024-09-12T07:20:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.243179827000001, 2024-09-12T07:20:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013681594, 2024-09-12T07:20:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.501634+0000", 2024-09-12T07:20:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.501634+0000", 2024-09-12T07:20:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.501636+0000", 2024-09-12T07:20:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.668e-06 2024-09-12T07:20:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.508127+0000", 2024-09-12T07:20:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064905789999999998 2024-09-12T07:20:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.508152+0000", 2024-09-12T07:20:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5208e-05 2024-09-12T07:20:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.508165+0000", 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2677000000000001e-05 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.508243+0000", 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8857999999999996e-05 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.508274+0000", 2024-09-12T07:20:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0954999999999998e-05 2024-09-12T07:20:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.515249+0000", 2024-09-12T07:20:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069747919999999996 2024-09-12T07:20:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.515316+0000", 2024-09-12T07:20:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6857e-05 2024-09-12T07:20:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:88838 3.2 3:407ead74:::rbd_data.11607a56846.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 2220620~4647] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:13.589368+0000", 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.155446133, 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012610503, 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 88838 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.589368+0000", 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.589368+0000", 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.589369+0000", 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.268e-06 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.590751+0000", 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001381125 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.590780+0000", 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9173e-05 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.590794+0000", 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4062e-05 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.590878+0000", 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3963999999999998e-05 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.591062+0000", 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018438300000000001 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.591511+0000", 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044836300000000002 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.599382+0000", 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078715820000000002 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.601849+0000", 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024668369999999999 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.601914+0000", 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5306000000000001e-05 2024-09-12T07:20:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.601979+0000", 2024-09-12T07:20:26.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.444e-05 2024-09-12T07:20:26.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.762 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.762 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:88880 3.18 3:1b2fb8d6:::rbd_data.11607a56846.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 2129920~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-12T07:20:26.762 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:13.609971+0000", 2024-09-12T07:20:26.763 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.134842902999999, 2024-09-12T07:20:26.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011637115, 2024-09-12T07:20:26.763 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.763 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.763 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:26.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 88880 2024-09-12T07:20:26.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.763 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.609971+0000", 2024-09-12T07:20:26.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.609971+0000", 2024-09-12T07:20:26.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.609972+0000", 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.046e-06 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.611254+0000", 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001281882 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.611275+0000", 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0791000000000001e-05 2024-09-12T07:20:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.611284+0000", 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0899999999999994e-06 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.611366+0000", 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1933000000000003e-05 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.611539+0000", 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172771 2024-09-12T07:20:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.611866+0000", 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032747000000000001 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.620730+0000", 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088641009999999992 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.621487+0000", 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00075658099999999999 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.621557+0000", 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9521000000000001e-05 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:13.621609+0000", 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1928999999999997e-05 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:90123 3.2 3:407ead74:::rbd_data.11607a56846.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-12T07:20:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:14.259915+0000", 2024-09-12T07:20:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.484898898000001, 2024-09-12T07:20:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010349486999999999, 2024-09-12T07:20:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 90123 2024-09-12T07:20:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259915+0000", 2024-09-12T07:20:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259915+0000", 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259916+0000", 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0279999999999999e-06 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259921+0000", 2024-09-12T07:20:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8930000000000004e-06 2024-09-12T07:20:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259934+0000", 2024-09-12T07:20:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2381000000000001e-05 2024-09-12T07:20:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259949+0000", 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5729000000000001e-05 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.259994+0000", 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4860000000000001e-05 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.260047+0000", 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2468e-05 2024-09-12T07:20:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.261523+0000", 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014767300000000001 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.270189+0000", 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086655619999999999 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.270200+0000", 2024-09-12T07:20:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1345999999999999e-05 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.270233+0000", 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2141000000000003e-05 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.270265+0000", 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2348999999999998e-05 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:90226 3.8 3:15b350ab:::rbd_data.11607a56846.000000000000001c:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:14.355994+0000", 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.388819955000001, 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012285943000000001, 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 90226 2024-09-12T07:20:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.355994+0000", 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.355994+0000", 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.355996+0000", 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.367e-06 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.356002+0000", 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0419999999999999e-06 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.356015+0000", 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3543000000000001e-05 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.356023+0000", 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6450000000000005e-06 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.356075+0000", 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460999999999998e-05 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.356147+0000", 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1612999999999994e-05 2024-09-12T07:20:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.358786+0000", 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002639045 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.368142+0000", 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093560329999999997 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.368162+0000", 2024-09-12T07:20:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0343000000000002e-05 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.368225+0000", 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2069999999999994e-05 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:14.368280+0000", 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5781e-05 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97197 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3457024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.688930+0000", 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.0558839790000007, 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093541869999999999, 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97197 2024-09-12T07:20:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688930+0000", 2024-09-12T07:20:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688930+0000", 2024-09-12T07:20:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688931+0000", 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0699999999999999e-07 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688934+0000", 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8700000000000002e-06 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688939+0000", 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8609999999999997e-06 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688948+0000", 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2019999999999993e-06 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698105+0000", 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.009156951 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698196+0000", 2024-09-12T07:20:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0845000000000002e-05 2024-09-12T07:20:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698285+0000", 2024-09-12T07:20:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8251000000000006e-05 2024-09-12T07:20:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97198 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.688955+0000", 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.0558590080000005, 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094347690000000008, 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97198 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688955+0000", 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688955+0000", 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688956+0000", 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9700000000000003e-07 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688957+0000", 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4720000000000001e-06 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688961+0000", 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8399999999999997e-06 2024-09-12T07:20:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688964+0000", 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7e-06 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698310+0000", 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093460009999999996 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698350+0000", 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0630000000000002e-05 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698390+0000", 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9829000000000001e-05 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97199 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.688967+0000", 2024-09-12T07:20:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.0558472670000008, 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096352060000000003, 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97199 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688967+0000", 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688967+0000", 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688967+0000", 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1e-07 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688975+0000", 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1149999999999994e-06 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688981+0000", 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2429999999999996e-06 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688983+0000", 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5749999999999999e-06 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698425+0000", 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094418430000000001 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698516+0000", 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0897999999999996e-05 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698602+0000", 2024-09-12T07:20:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6322000000000005e-05 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97200 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.688987+0000", 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.0558275570000006, 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0097196250000000008, 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97200 2024-09-12T07:20:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688987+0000", 2024-09-12T07:20:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688987+0000", 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688987+0000", 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.36e-07 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688988+0000", 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2300000000000001e-06 2024-09-12T07:20:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688993+0000", 2024-09-12T07:20:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8420000000000004e-06 2024-09-12T07:20:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688996+0000", 2024-09-12T07:20:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4509999999999999e-06 2024-09-12T07:20:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698627+0000", 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096317650000000005 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698668+0000", 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0819999999999999e-05 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698706+0000", 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8281000000000002e-05 2024-09-12T07:20:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97201 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.688999+0000", 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.0558155469999999, 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099215780000000003, 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97201 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688999+0000", 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688999+0000", 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.688999+0000", 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.16e-07 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689000+0000", 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0240000000000001e-06 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689003+0000", 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0769999999999999e-06 2024-09-12T07:20:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689006+0000", 2024-09-12T07:20:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.491e-06 2024-09-12T07:20:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698740+0000", 2024-09-12T07:20:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0097341709999999998 2024-09-12T07:20:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698831+0000", 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0821000000000003e-05 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698920+0000", 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9777999999999993e-05 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97202 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.689009+0000", 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.0558056029999996, 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010021258, 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97202 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689009+0000", 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689009+0000", 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689009+0000", 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4400000000000001e-07 2024-09-12T07:20:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689010+0000", 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0759999999999999e-06 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689013+0000", 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9409999999999999e-06 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689016+0000", 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.253e-06 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698950+0000", 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099334829999999999 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.698991+0000", 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1619000000000003e-05 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699030+0000", 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8642e-05 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97203 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.689021+0000", 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.0557934499999995, 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010221196, 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97203 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689021+0000", 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689021+0000", 2024-09-12T07:20:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689021+0000", 2024-09-12T07:20:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5699999999999998e-07 2024-09-12T07:20:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689022+0000", 2024-09-12T07:20:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0550000000000001e-06 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689030+0000", 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5449999999999998e-06 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689034+0000", 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8430000000000003e-06 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699067+0000", 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010033432 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699158+0000", 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0420000000000005e-05 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699242+0000", 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4543999999999996e-05 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97204 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.689037+0000", 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.0557769720000003, 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010307439, 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97204 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689037+0000", 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689037+0000", 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689038+0000", 2024-09-12T07:20:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.34e-07 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689039+0000", 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1939999999999999e-06 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689042+0000", 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2559999999999998e-06 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689045+0000", 2024-09-12T07:20:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4760000000000001e-06 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699267+0000", 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010222825 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699306+0000", 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8463000000000001e-05 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699345+0000", 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8890999999999997e-05 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:97205 3.8 3:12271cdf:::rbd_data.1160b01e9f21.0000000000000021:12 [read 4145152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:17.689048+0000", 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.0557659009999991, 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010502008, 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97205 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689048+0000", 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689048+0000", 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689049+0000", 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.23e-07 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689050+0000", 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689053+0000", 2024-09-12T07:20:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0089999999999999e-06 2024-09-12T07:20:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.689055+0000", 2024-09-12T07:20:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4320000000000002e-06 2024-09-12T07:20:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699371+0000", 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010316201000000001 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699465+0000", 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3599999999999998e-05 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:17.699550+0000", 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5477999999999997e-05 2024-09-12T07:20:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:100373 3.8 3:16d49e60:::rbd_data.1160d60c02f0.0000000000000005:15 [read 1376256~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:21.479532+0000", 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.2652820919999996, 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095017790000000001, 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100373 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479532+0000", 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479532+0000", 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479533+0000", 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3000000000000001e-07 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479535+0000", 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479543+0000", 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1310000000000006e-06 2024-09-12T07:20:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479550+0000", 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1409999999999999e-06 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.488839+0000", 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092886740000000002 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.488924+0000", 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5410999999999999e-05 2024-09-12T07:20:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489034+0000", 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010980900000000001 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:100374 3.8 3:16d49e60:::rbd_data.1160d60c02f0.0000000000000005:15 [read 1605632~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:21.479558+0000", 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.2652560829999997, 2024-09-12T07:20:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0097197859999999994, 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100374 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479558+0000", 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479558+0000", 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479559+0000", 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2500000000000001e-07 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479561+0000", 2024-09-12T07:20:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2919999999999998e-06 2024-09-12T07:20:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479569+0000", 2024-09-12T07:20:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3250000000000008e-06 2024-09-12T07:20:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479576+0000", 2024-09-12T07:20:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9480000000000002e-06 2024-09-12T07:20:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489081+0000", 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095043899999999997 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489167+0000", 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6790000000000001e-05 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489278+0000", 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110616 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:100375 3.8 3:16d49e60:::rbd_data.1160d60c02f0.0000000000000005:15 [read 1835008~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:21.479586+0000", 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.2652286830000001, 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099179139999999999, 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100375 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479586+0000", 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479586+0000", 2024-09-12T07:20:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479586+0000", 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479588+0000", 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.024e-06 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479596+0000", 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1680000000000004e-06 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479603+0000", 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6490000000000002e-06 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489321+0000", 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0097178560000000004 2024-09-12T07:20:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489400+0000", 2024-09-12T07:20:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9059000000000003e-05 2024-09-12T07:20:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489504+0000", 2024-09-12T07:20:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103705 2024-09-12T07:20:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:100376 3.8 3:16d49e60:::rbd_data.1160d60c02f0.0000000000000005:15 [read 2064384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-12T07:20:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:21.479611+0000", 2024-09-12T07:20:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.2652031060000004, 2024-09-12T07:20:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010113561, 2024-09-12T07:20:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100376 2024-09-12T07:20:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479611+0000", 2024-09-12T07:20:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479611+0000", 2024-09-12T07:20:26.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479612+0000", 2024-09-12T07:20:26.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5499999999999998e-07 2024-09-12T07:20:26.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479614+0000", 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.934e-06 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479622+0000", 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2179999999999999e-06 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.479628+0000", 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5830000000000003e-06 2024-09-12T07:20:26.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489534+0000", 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099051980000000005 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489615+0000", 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1787999999999997e-05 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.489725+0000", 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109385 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:100438 3.2 3:46b60448:::rbd_data.11606e757b33.000000000000005e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:21.550725+0000", 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.1940891530000002, 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010300614, 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100438 2024-09-12T07:20:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550725+0000", 2024-09-12T07:20:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550725+0000", 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550726+0000", 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550731+0000", 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9119999999999997e-06 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550742+0000", 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1233000000000001e-05 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550747+0000", 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0760000000000002e-06 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550801+0000", 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4012999999999999e-05 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.550882+0000", 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0953999999999998e-05 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.552616+0000", 2024-09-12T07:20:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017335079999999999 2024-09-12T07:20:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.560932+0000", 2024-09-12T07:20:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083159500000000008 2024-09-12T07:20:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.560968+0000", 2024-09-12T07:20:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6254000000000002e-05 2024-09-12T07:20:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.561000+0000", 2024-09-12T07:20:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1506e-05 2024-09-12T07:20:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:21.561026+0000", 2024-09-12T07:20:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6250000000000001e-05 2024-09-12T07:20:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:26.851 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:26.853 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:26.853 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:26.854+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:26.890 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-12T07:20:26.953 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:26.954+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:26.997 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:26.998 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:27.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:27.054 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:27.055+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:27.153 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:27.153 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:27.153 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:27.153 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:27.154 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:27.155 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:27.156+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:27.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:27.256 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:27.258+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.0 to 1 (10000) 2024-09-12T07:20:27.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-12T07:20:27.335 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:27.335 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:27.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:27.335 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:27.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.335 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 29.722411617999999, 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.339 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 24.586951362000001, 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:20:27.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 24.582724241000001, 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.344 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:20:27.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.348 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.854973586, 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:20:27.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:20:27.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.765996996999998, 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:20:27.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:20:27.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68853 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'43, mlcod=43'39)", 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:27.357 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.755004410000002, 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017793166999999999, 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580719+0000", 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4720000000000001e-06 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580732+0000", 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2926e-05 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580746+0000", 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3291e-05 2024-09-12T07:20:27.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580755+0000", 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6709999999999994e-06 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583766+0000", 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003010432 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583794+0000", 2024-09-12T07:20:27.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.811e-05 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598408+0000", 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014614472 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598511+0000", 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102793 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.734445666999999, 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:27.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:20:27.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:27.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:20:27.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.731245131000001, 2024-09-12T07:20:27.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:20:27.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.370 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.727999383, 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:27.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.725757552000001, 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69559 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'29, mlcod=41'27)", 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:27.379 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.651426504, 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018119258999999999, 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:27.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684297+0000", 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.192e-06 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691509+0000", 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072122239999999997 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691524+0000", 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4562999999999999e-05 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691533+0000", 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1069999999999994e-06 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691598+0000", 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4666999999999996e-05 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691620+0000", 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2092000000000001e-05 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702352+0000", 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010731785000000001 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702415+0000", 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3628999999999997e-05 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.383 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69560 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'31, mlcod=41'27)", 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.643860381, 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016047555000000002, 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691863+0000", 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0299999999999999e-07 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691897+0000", 2024-09-12T07:20:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4765999999999999e-05 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691926+0000", 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8413000000000001e-05 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691934+0000", 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3839999999999997e-06 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694739+0000", 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028043640000000002 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694773+0000", 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4514000000000002e-05 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707854+0000", 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013080591000000001 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707910+0000", 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.592e-05 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "pg_update_log_missing(3.1a epoch 43/24 rep_tid 362 entries 43'32 (0'0) error 3:5ec4a676:::rbd_data.1160f022dd61.0000000000000019:head by client.4448.0:69639 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 41'27)", 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.643332666999999, 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015537618, 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692497+0000", 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107154 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692519+0000", 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1043000000000002e-05 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692542+0000", 2024-09-12T07:20:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3866000000000002e-05 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702496+0000", 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0099539169999999996 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707927+0000", 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005431038 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.634650348000001, 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:20:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:20:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:20:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.630822259999999, 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89144 3.a e53/24 3:54996542:::rbd_data.11607a56846.0000000000000001:head v 53'83, mlcod=53'81)", 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.642665506, 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018298937000000001, 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693059+0000", 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.565e-06 2024-09-12T07:20:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696238+0000", 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0031783269999999999 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696265+0000", 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6982e-05 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696279+0000", 2024-09-12T07:20:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4438e-05 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696366+0000", 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6978000000000006e-05 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696397+0000", 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0406999999999999e-05 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711297+0000", 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014900689999999999 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711356+0000", 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.855e-05 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89145 3.13 e53/24 3:ca7c3842:::rbd_data.11607a56846.0000000000000002:head v 53'202, mlcod=53'199)", 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.641634162999999, 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019835603, 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694091+0000", 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4990000000000001e-06 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697616+0000", 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035256749999999998 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697635+0000", 2024-09-12T07:20:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8593999999999998e-05 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697646+0000", 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1436e-05 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697717+0000", 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0679000000000005e-05 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697748+0000", 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0723999999999999e-05 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713886+0000", 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016138550000000002 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713924+0000", 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7446e-05 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.64890947899999996, 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:20:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.637095835, 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:20:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:20:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:20:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.63575978799999999, 2024-09-12T07:20:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:20:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:20:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.422 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:27.424 INFO:teuthology.orchestra.run.smithi046.stdout:2694 writesame 0xe08e0a4 thru 0xe097fb6 data_size 0x1(0x9f13 bytes) 2024-09-12T07:20:27.424 INFO:teuthology.orchestra.run.smithi046.stdout:2695 punch from 0x9d1a4b to 0x9d7c37, (0x61ec bytes) 2024-09-12T07:20:27.424 INFO:teuthology.orchestra.run.smithi046.stdout:2696 read 0x2033a09 thru 0x20374b0 (0x3aa8 bytes) 2024-09-12T07:20:27.424 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:27.424 INFO:teuthology.orchestra.run.smithi046.stdout:2699 writesame 0x598a1ef thru 0x59982e4 data_size 0x5(0xe0f6 bytes) 2024-09-12T07:20:27.425 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:27.425+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:27.425 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:27.426 INFO:teuthology.orchestra.run.smithi046.stdout:2701 punch from 0x2ef9f60 to 0x2f02fa9, (0x9049 bytes) 2024-09-12T07:20:27.426 INFO:teuthology.orchestra.run.smithi046.stdout:2702 write 0xac56db5 thru 0xac5e854 (0x7aa0 bytes) 2024-09-12T07:20:27.453 INFO:teuthology.orchestra.run.smithi046.stdout:2703 write 0x3711ebb thru 0x37160b8 (0x41fe bytes) 2024-09-12T07:20:27.475 INFO:teuthology.orchestra.run.smithi046.stdout:2704 read 0x4a0ad6e thru 0x4a156c1 (0xa954 bytes) 2024-09-12T07:20:27.477 INFO:teuthology.orchestra.run.smithi046.stdout:2707 writesame 0x8b6f06b thru 0x8b72d17 data_size 0x7(0x3cad bytes) 2024-09-12T07:20:27.498 INFO:teuthology.orchestra.run.smithi046.stdout:2708 read 0x9639f58 thru 0x963e4ae (0x4557 bytes) 2024-09-12T07:20:27.499 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:27.499 INFO:teuthology.orchestra.run.smithi046.stdout:2711 read 0x1492111 thru 0x149bd68 (0x9c58 bytes) 2024-09-12T07:20:27.500 INFO:teuthology.orchestra.run.smithi046.stdout:2712 read 0x2ec1dad thru 0x2ec892b (0x6b7f bytes) 2024-09-12T07:20:27.501 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:27.502 INFO:teuthology.orchestra.run.smithi046.stdout:2714 write 0xd55475c thru 0xd55d374 (0x8c19 bytes) 2024-09-12T07:20:27.510 INFO:teuthology.orchestra.run.smithi046.stdout:2716 write 0x137270 thru 0x1451ff (0xdf90 bytes) 2024-09-12T07:20:27.524 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:27.526+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:27.592 INFO:teuthology.orchestra.run.smithi046.stdout:2720 write 0x5ce948d thru 0x5cea959 (0x14cd bytes) 2024-09-12T07:20:27.601 INFO:teuthology.orchestra.run.smithi046.stdout:2721 write 0x8e2251c thru 0x8e2a1ea (0x7ccf bytes) 2024-09-12T07:20:27.617 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:27.617 INFO:teuthology.orchestra.run.smithi046.stdout:2726 read 0xcf8693 thru 0xd00e49 (0x87b7 bytes) 2024-09-12T07:20:27.617 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:126891 3.1c 3:3fc13148:::rbd_data.1160906c0716.0000000000000086:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 1048576 write_size 1048576,write 884736~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:27.616713+0000", 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0025934389999999999, 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002629756, 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for sub ops", 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 126891 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.616713+0000", 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.616713+0000", 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.616714+0000", 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6400000000000002e-07 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.616722+0000", 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1850000000000004e-06 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.616732+0000", 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6819999999999998e-06 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.616739+0000", 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0400000000000004e-06 2024-09-12T07:20:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.618081+0000", 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0013419619999999999 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.618180+0000", 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8975999999999996e-05 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 4", 2024-09-12T07:20:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.618331+0000", 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015105899999999999 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.618971+0000", 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00063963400000000004 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop_reply(client.4448.0:126890 3.1c e66/24 ondisk, result = 0)", 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:27.619271+0000", 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 3.5639999999999998e-05, 2024-09-12T07:20:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000148157, 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.619271+0000", 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.619271+0000", 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.619272+0000", 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.052e-06 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.619276+0000", 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8820000000000003e-06 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.619287+0000", 2024-09-12T07:20:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0508e-05 2024-09-12T07:20:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.619292+0000", 2024-09-12T07:20:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-06 2024-09-12T07:20:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.619337+0000", 2024-09-12T07:20:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4916999999999997e-05 2024-09-12T07:20:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:27.619342+0000", 2024-09-12T07:20:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9579999999999998e-06 2024-09-12T07:20:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T07:20:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 2 2024-09-12T07:20:27.626 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:2727 punch from 0x6e3d4ba to 0x6e42385, (0x4ecb bytes) 2024-09-12T07:20:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:2729 read 0x36b5b1b thru 0x36bf14e (0x9634 bytes) 2024-09-12T07:20:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:2730 read 0x3a3cc88 thru 0x3a4a0cc (0xd445 bytes) 2024-09-12T07:20:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:2734 read 0x929829e thru 0x929ed6e (0x6ad1 bytes) 2024-09-12T07:20:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:2735 punch from 0x637e780 to 0x638345d, (0x4cdd bytes) 2024-09-12T07:20:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:2736 read 0xce8d97c thru 0xce91e1e (0x44a3 bytes) 2024-09-12T07:20:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:2737 write 0x9d7daef thru 0x9d83640 (0x5b52 bytes) 2024-09-12T07:20:27.628 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:27.629+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:27.629 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:27.650 INFO:teuthology.orchestra.run.smithi046.stdout:2738 punch from 0x29fa4ce to 0x29fce2a, (0x295c bytes) 2024-09-12T07:20:27.650 INFO:teuthology.orchestra.run.smithi046.stdout:2739 read 0x1e7860d thru 0x1e7df88 (0x597c bytes) 2024-09-12T07:20:27.651 INFO:teuthology.orchestra.run.smithi046.stdout:2740 writesame 0xc99cc98 thru 0xc9a9334 data_size 0x1(0xc69d bytes) 2024-09-12T07:20:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:2741 write 0x1a335a4 thru 0x1a3ccd3 (0x9730 bytes) 2024-09-12T07:20:27.728 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:27.730+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:27.752 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:27.753 INFO:teuthology.orchestra.run.smithi046.stdout:2744 read 0xa7b88eb thru 0xa7c0930 (0x8046 bytes) 2024-09-12T07:20:27.753 INFO:teuthology.orchestra.run.smithi046.stdout:2745 punch from 0x6a5f486 to 0x6a6903a, (0x9bb4 bytes) 2024-09-12T07:20:27.753 INFO:teuthology.orchestra.run.smithi046.stdout:2746 writesame 0x2a1f47b thru 0x2a2cb9f data_size 0xb(0xd725 bytes) 2024-09-12T07:20:27.827 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:27.827 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:2747 trunc from 0xe097fb7 to 0x90cc3d1 2024-09-12T07:20:27.828 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:27.830+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:27.838 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:2748 read 0x8806571 thru 0x880b8ae (0x533e bytes) 2024-09-12T07:20:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:2749 write 0xf2019e thru 0xf2e642 (0xe4a5 bytes) 2024-09-12T07:20:27.929 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:27.931+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:28.006 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.007 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.012 INFO:teuthology.orchestra.run.smithi046.stdout:2753 write 0x98a8c65 thru 0x98ac6dd (0x3a79 bytes) 2024-09-12T07:20:28.013 INFO:teuthology.orchestra.run.smithi046.stdout:2755 read 0x4bae1f4 thru 0x4bb66c4 (0x84d1 bytes) 2024-09-12T07:20:28.014 INFO:teuthology.orchestra.run.smithi046.stdout:2757 read 0x4416590 thru 0x441f47c (0x8eed bytes) 2024-09-12T07:20:28.015 INFO:teuthology.orchestra.run.smithi046.stdout:2758 read 0xf9215f thru 0xf93a5e (0x1900 bytes) 2024-09-12T07:20:28.016 INFO:teuthology.orchestra.run.smithi046.stdout:2759 writesame 0x8a436d3 thru 0x8a4e776 data_size 0x26(0xb0a4 bytes) 2024-09-12T07:20:28.030 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:20:28.032 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:28.032+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:28.033 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.551877066999999, 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:20:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.551850186999999, 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:20:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.551826167000002, 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:20:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:20:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:20:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.551798520999998, 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:20:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:20:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:20:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:20:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:20:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:20:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.551775059000001, 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:20:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.551752531999998, 2024-09-12T07:20:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.551730728999999, 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:20:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.545096949000001, 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:20:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:20:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:20:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:20:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:20:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:20:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.545076941000001, 2024-09-12T07:20:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.538593908999999, 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:20:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.538558622, 2024-09-12T07:20:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:20:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:20:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.538535989, 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:20:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.538481934, 2024-09-12T07:20:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:20:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.538462993, 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:20:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.538451056, 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:20:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:20:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:20:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:20:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:20:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:20:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.538439441000001, 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:20:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.531426833000001, 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:20:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:20:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.531403118, 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:20:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.531272261000002, 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 24.567601471, 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:20:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:20:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:20:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:20:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:20:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:20:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:20:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:28.126 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:28.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:28.129 INFO:teuthology.orchestra.run.smithi046.stdout:2760 trunc from 0x98ac6de to 0xd8ab6bb 2024-09-12T07:20:28.129 INFO:teuthology.orchestra.run.smithi046.stdout:2761 read 0xb6dd8fe thru 0xb6e4225 (0x6928 bytes) 2024-09-12T07:20:28.129 INFO:teuthology.orchestra.run.smithi046.stdout:2762 read 0x4906880 thru 0x490c597 (0x5d18 bytes) 2024-09-12T07:20:28.129 INFO:teuthology.orchestra.run.smithi046.stdout:2765 read 0x82aa8a7 thru 0x82b1389 (0x6ae3 bytes) 2024-09-12T07:20:28.129 INFO:teuthology.orchestra.run.smithi046.stdout:2766 writesame 0x268dce8 thru 0x269d7c7 data_size 0x20(0xfae0 bytes) 2024-09-12T07:20:28.129 INFO:teuthology.orchestra.run.smithi046.stdout:2767 trunc from 0xd8ab6bb to 0x4129255 2024-09-12T07:20:28.172 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:28.173+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:28.182 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:28.182 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:28.199 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:28.199 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:28.215 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:28.232 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:28.232 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:28.232 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:28.234+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:28.248 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:28.249 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:28.266 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:28.266 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:28.278 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:28.278 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:28.283 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.1 to 1 (10000) 2024-09-12T07:20:28.283 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:28.283 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:28.288 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:28.292 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:20:28.297 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-12T07:20:28.301 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:28.372 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:28.374+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:28.383 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.399 INFO:teuthology.orchestra.run.smithi046.stdout:2769 write 0xc2f1959 thru 0xc2f92f6 (0x799e bytes) 2024-09-12T07:20:28.406 INFO:teuthology.orchestra.run.smithi046.stdout:2770 read 0x78b9007 thru 0x78bfe8d (0x6e87 bytes) 2024-09-12T07:20:28.406 INFO:teuthology.orchestra.run.smithi046.stdout:2772 read 0x428ff13 thru 0x429400f (0x40fd bytes) 2024-09-12T07:20:28.415 INFO:teuthology.orchestra.run.smithi046.stdout:2773 write 0xe78066a thru 0xe79044c (0xfde3 bytes) 2024-09-12T07:20:28.427 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.427 INFO:teuthology.orchestra.run.smithi046.stdout:2775 write 0x56db447 thru 0x56e0231 (0x4deb bytes) 2024-09-12T07:20:28.433 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:28.435+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:28.434 INFO:teuthology.orchestra.run.smithi046.stdout:2776 write 0x5f5375b thru 0x5f5eaf1 (0xb397 bytes) 2024-09-12T07:20:28.444 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.444 INFO:teuthology.orchestra.run.smithi046.stdout:2779 read 0x9efe253 thru 0x9f0cbce (0xe97c bytes) 2024-09-12T07:20:28.444 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.444 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.444 INFO:teuthology.orchestra.run.smithi046.stdout:2782 write 0xa28e71e thru 0xa29232c (0x3c0f bytes) 2024-09-12T07:20:28.445 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:28.445 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:28.449 INFO:teuthology.orchestra.run.smithi046.stdout:2783 read 0x509187 thru 0x518b40 (0xf9ba bytes) 2024-09-12T07:20:28.451 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.451 INFO:teuthology.orchestra.run.smithi046.stdout:2787 write 0xb2406f8 thru 0xb24afde (0xa8e7 bytes) 2024-09-12T07:20:28.454 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:28.460 INFO:teuthology.orchestra.run.smithi046.stdout:2789 read 0xb1f7ede thru 0xb1feed8 (0x6ffb bytes) 2024-09-12T07:20:28.460 INFO:teuthology.orchestra.run.smithi046.stdout:2791 read 0xe54318b thru 0xe54f21c (0xc092 bytes) 2024-09-12T07:20:28.461 INFO:teuthology.orchestra.run.smithi046.stdout:2793 punch from 0x2263bd4 to 0x2270dd5, (0xd201 bytes) 2024-09-12T07:20:28.461 INFO:teuthology.orchestra.run.smithi046.stdout:2794 read 0x6398666 thru 0x639b056 (0x29f1 bytes) 2024-09-12T07:20:28.461 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.461 INFO:teuthology.orchestra.run.smithi046.stdout:2796 write 0xd17ceb7 thru 0xd185821 (0x896b bytes) 2024-09-12T07:20:28.469 INFO:teuthology.orchestra.run.smithi046.stdout:2797 write 0x3dfe5ab thru 0x3e0a7c1 (0xc217 bytes) 2024-09-12T07:20:28.477 INFO:teuthology.orchestra.run.smithi046.stdout:2798 writesame 0xe824bda thru 0xe8277b5 data_size 0xe(0x2bdc bytes) 2024-09-12T07:20:28.477 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.478 INFO:teuthology.orchestra.run.smithi046.stdout:2802 write 0x12df588 thru 0x12e910f (0x9b88 bytes) 2024-09-12T07:20:28.492 INFO:teuthology.orchestra.run.smithi046.stdout:2804 write 0xcc18fac thru 0xcc1b61c (0x2671 bytes) 2024-09-12T07:20:28.496 INFO:teuthology.orchestra.run.smithi046.stdout:2805 write 0x334d52 thru 0x34073c (0xb9eb bytes) 2024-09-12T07:20:28.522 INFO:teuthology.orchestra.run.smithi046.stdout:2807 read 0x930091b thru 0x930833d (0x7a23 bytes) 2024-09-12T07:20:28.523 INFO:teuthology.orchestra.run.smithi046.stdout:2808 flatten 2024-09-12T07:20:28.534 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:28.535+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:28.634 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:28.636+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:28.659 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:28.659 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:28.671 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:28.716 INFO:teuthology.orchestra.run.smithi046.stdout:2811 write 0xb584b43 thru 0xb58a113 (0x55d1 bytes) 2024-09-12T07:20:28.723 INFO:teuthology.orchestra.run.smithi046.stdout:2812 read 0x21c4e0d thru 0x21cd976 (0x8b6a bytes) 2024-09-12T07:20:28.724 INFO:teuthology.orchestra.run.smithi046.stdout:2813 read 0xc757f71 thru 0xc765c62 (0xdcf2 bytes) 2024-09-12T07:20:28.725 INFO:teuthology.orchestra.run.smithi046.stdout:2814 punch from 0x4e916bb to 0x4e93b32, (0x2477 bytes) 2024-09-12T07:20:28.725 INFO:teuthology.orchestra.run.smithi046.stdout:2815 write 0xbf04555 thru 0xbf08fb9 (0x4a65 bytes) 2024-09-12T07:20:28.735 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:28.736+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:28.740 INFO:teuthology.orchestra.run.smithi046.stdout:2816 trunc from 0xe8277b6 to 0x5acf015 2024-09-12T07:20:28.762 INFO:teuthology.orchestra.run.smithi046.stdout:2817 read 0x2c52dc8 thru 0x2c5796f (0x4ba8 bytes) 2024-09-12T07:20:28.762 INFO:teuthology.orchestra.run.smithi046.stdout:2818 read 0xb27e91 thru 0xb31678 (0x97e8 bytes) 2024-09-12T07:20:28.763 INFO:teuthology.orchestra.run.smithi046.stdout:2819 punch from 0x12da7ae to 0x12db258, (0xaaa bytes) 2024-09-12T07:20:28.774 INFO:teuthology.orchestra.run.smithi046.stdout:2820 write 0xab55702 thru 0xab5c35e (0x6c5d bytes) 2024-09-12T07:20:28.783 INFO:teuthology.orchestra.run.smithi046.stdout:2821 read 0xa78978e thru 0xa78e6a4 (0x4f17 bytes) 2024-09-12T07:20:28.783 INFO:teuthology.orchestra.run.smithi046.stdout:2822 punch from 0x3a952a0 to 0x3a9a7ce, (0x552e bytes) 2024-09-12T07:20:28.783 INFO:teuthology.orchestra.run.smithi046.stdout:2823 write 0x10eb9fe thru 0x10faf55 (0xf558 bytes) 2024-09-12T07:20:28.787 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.787 INFO:teuthology.orchestra.run.smithi046.stdout:2826 read 0x35397e5 thru 0x35441b4 (0xa9d0 bytes) 2024-09-12T07:20:28.787 INFO:teuthology.orchestra.run.smithi046.stdout:2827 writesame 0x71f53bd thru 0x71fb72f data_size 0x1(0x6373 bytes) 2024-09-12T07:20:28.793 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:28.793 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:28.796 INFO:teuthology.orchestra.run.smithi046.stdout:2828 read 0xe1e5b2 thru 0xe20955 (0x23a4 bytes) 2024-09-12T07:20:28.799 INFO:teuthology.orchestra.run.smithi046.stdout:2829 trunc from 0xab5c35f to 0x8209f67 2024-09-12T07:20:28.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:28.814 INFO:teuthology.orchestra.run.smithi046.stdout:2830 writesame 0x89a6c13 thru 0x89af67c data_size 0x7(0x8a6a bytes) 2024-09-12T07:20:28.814 INFO:teuthology.orchestra.run.smithi046.stdout:2831 write 0x49b289a thru 0x49b7c63 (0x53ca bytes) 2024-09-12T07:20:28.818 INFO:teuthology.orchestra.run.smithi046.stdout:2832 read 0x1ca9ad2 thru 0x1cb9181 (0xf6b0 bytes) 2024-09-12T07:20:28.829 INFO:teuthology.orchestra.run.smithi046.stdout:2834 write 0xc6f9dc8 thru 0xc6fabdb (0xe14 bytes) 2024-09-12T07:20:28.833 INFO:teuthology.orchestra.run.smithi046.stdout:2835 punch from 0xa4a1814 to 0xa4adb96, (0xc382 bytes) 2024-09-12T07:20:28.833 INFO:teuthology.orchestra.run.smithi046.stdout:2837 write 0x836fa44 thru 0x837d0f0 (0xd6ad bytes) 2024-09-12T07:20:28.843 INFO:teuthology.orchestra.run.smithi046.stdout:2838 write 0x9697e08 thru 0x96a2121 (0xa31a bytes) 2024-09-12T07:20:28.844 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:28.845+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:28.852 INFO:teuthology.orchestra.run.smithi046.stdout:2839 writesame 0xb4966c4 thru 0xb499663 data_size 0x20(0x2fa0 bytes) 2024-09-12T07:20:28.855 INFO:teuthology.orchestra.run.smithi046.stdout:2840 punch from 0x8de8e75 to 0x8deabfc, (0x1d87 bytes) 2024-09-12T07:20:28.862 INFO:teuthology.orchestra.run.smithi046.stdout:2841 trunc from 0xc6fabdc to 0x6e805c2 2024-09-12T07:20:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:2842 read 0x53a4ab5 thru 0x53a7c7b (0x31c7 bytes) 2024-09-12T07:20:28.885 INFO:teuthology.orchestra.run.smithi046.stdout:2843 write 0x963cf8f thru 0x96409bc (0x3a2e bytes) 2024-09-12T07:20:28.889 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.889 INFO:teuthology.orchestra.run.smithi046.stdout:2845 punch from 0x18c8ab9 to 0x18ccfc8, (0x450f bytes) 2024-09-12T07:20:28.889 INFO:teuthology.orchestra.run.smithi046.stdout:2846 read 0x93ab368 thru 0x93b1448 (0x60e1 bytes) 2024-09-12T07:20:28.901 INFO:teuthology.orchestra.run.smithi046.stdout:2849 writesame 0xc865958 thru 0xc867a47 data_size 0x3e(0x20f0 bytes) 2024-09-12T07:20:28.901 INFO:teuthology.orchestra.run.smithi046.stdout:2850 read 0x6267637 thru 0x6270644 (0x900e bytes) 2024-09-12T07:20:28.901 INFO:teuthology.orchestra.run.smithi046.stdout:2851 read 0xacbaadc thru 0xacc7400 (0xc925 bytes) 2024-09-12T07:20:28.901 INFO:teuthology.orchestra.run.smithi046.stdout:2852 write 0x2a0d8c8 thru 0x2a1b60c (0xdd45 bytes) 2024-09-12T07:20:28.907 INFO:teuthology.orchestra.run.smithi046.stdout:2853 trunc from 0xc867a48 to 0x936bec6 2024-09-12T07:20:28.915 INFO:teuthology.orchestra.run.smithi046.stdout:2854 punch from 0x6b63b48 to 0x6b718d0, (0xdd88 bytes) 2024-09-12T07:20:28.915 INFO:teuthology.orchestra.run.smithi046.stdout:2856 read 0x7c35d68 thru 0x7c382c6 (0x255f bytes) 2024-09-12T07:20:28.916 INFO:teuthology.orchestra.run.smithi046.stdout:2857 read 0x30fd1c3 thru 0x3100910 (0x374e bytes) 2024-09-12T07:20:28.916 INFO:teuthology.orchestra.run.smithi046.stdout:2858 read 0x38ad2cf thru 0x38add2f (0xa61 bytes) 2024-09-12T07:20:28.916 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.922 INFO:teuthology.orchestra.run.smithi046.stdout:2860 write 0xa39a51d thru 0xa3a61c2 (0xbca6 bytes) 2024-09-12T07:20:28.930 INFO:teuthology.orchestra.run.smithi046.stdout:2862 write 0x84cbf1 thru 0x854363 (0x7773 bytes) 2024-09-12T07:20:28.937 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:28.939+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:28.941 INFO:teuthology.orchestra.run.smithi046.stdout:2863 write 0xd8589d1 thru 0xd86779f (0xedcf bytes) 2024-09-12T07:20:28.950 INFO:teuthology.orchestra.run.smithi046.stdout:2864 write 0x2300d4f thru 0x230256d (0x181f bytes) 2024-09-12T07:20:28.952 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:28.952 INFO:teuthology.orchestra.run.smithi046.stdout:2866 writesame 0x9d2368a thru 0x9d24698 data_size 0x1(0x100f bytes) 2024-09-12T07:20:28.954 INFO:teuthology.orchestra.run.smithi046.stdout:2867 read 0xbb3f4a5 thru 0xbb4b2f6 (0xbe52 bytes) 2024-09-12T07:20:28.959 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:28.959 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:28.959 INFO:teuthology.orchestra.run.smithi046.stdout:2868 trunc from 0xd8677a0 to 0x99d75c7 2024-09-12T07:20:28.969 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:2869 trunc from 0x99d75c7 to 0x3e4675d 2024-09-12T07:20:29.003 INFO:teuthology.orchestra.run.smithi046.stdout:2870 writesame 0x6b82e72 thru 0x6b929f4 data_size 0x1f(0xfb83 bytes) 2024-09-12T07:20:29.051 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.053 INFO:teuthology.orchestra.run.smithi046.stdout:2874 writesame 0x8e12d61 thru 0x8e1cf0e data_size 0x5(0xa1ae bytes) 2024-09-12T07:20:29.053 INFO:teuthology.orchestra.run.smithi046.stdout:2875 punch from 0x6842cb6 to 0x684d0dc, (0xa426 bytes) 2024-09-12T07:20:29.054 INFO:teuthology.orchestra.run.smithi046.stdout:2876 trunc from 0x8e1cf0f to 0x3432e71 2024-09-12T07:20:29.054 INFO:teuthology.orchestra.run.smithi046.stdout:2877 write 0x97b1625 thru 0x97b9da9 (0x8785 bytes) 2024-09-12T07:20:29.054 INFO:teuthology.orchestra.run.smithi046.stdout:2880 read 0x5d2e07f thru 0x5d38ec1 (0xae43 bytes) 2024-09-12T07:20:29.054 INFO:teuthology.orchestra.run.smithi046.stdout:2882 read 0x5c91a70 thru 0x5c92473 (0xa04 bytes) 2024-09-12T07:20:29.054 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.054 INFO:teuthology.orchestra.run.smithi046.stdout:2884 read 0x66f5172 thru 0x66f8124 (0x2fb3 bytes) 2024-09-12T07:20:29.054 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.054 INFO:teuthology.orchestra.run.smithi046.stdout:2887 punch from 0x16d941d to 0x16e09cc, (0x75af bytes) 2024-09-12T07:20:29.054 INFO:teuthology.orchestra.run.smithi046.stdout:2888 punch from 0x44b31e0 to 0x44ba55a, (0x737a bytes) 2024-09-12T07:20:29.054 INFO:teuthology.orchestra.run.smithi046.stdout:2889 write 0x5c08376 thru 0x5c1665d (0xe2e8 bytes) 2024-09-12T07:20:29.054 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:29.054+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:29.057 INFO:teuthology.orchestra.run.smithi046.stdout:2890 writesame 0x888349b thru 0x888ccd1 data_size 0x3(0x9837 bytes) 2024-09-12T07:20:29.075 INFO:teuthology.orchestra.run.smithi046.stdout:2891 write 0xacfcd23 thru 0xad06a20 (0x9cfe bytes) 2024-09-12T07:20:29.083 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.083 INFO:teuthology.orchestra.run.smithi046.stdout:2894 writesame 0x2664880 thru 0x2665d5b data_size 0x1e(0x14dc bytes) 2024-09-12T07:20:29.084 INFO:teuthology.orchestra.run.smithi046.stdout:2896 read 0x96073df thru 0x960cdec (0x5a0e bytes) 2024-09-12T07:20:29.097 INFO:teuthology.orchestra.run.smithi046.stdout:2897 trunc from 0xad06a21 to 0x25fd64 2024-09-12T07:20:29.121 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.121 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.121 INFO:teuthology.orchestra.run.smithi046.stdout:2902 read 0x231cac thru 0x2351ce (0x3523 bytes) 2024-09-12T07:20:29.122 INFO:teuthology.orchestra.run.smithi046.stdout:2903 read 0x939af thru 0x9ae49 (0x749b bytes) 2024-09-12T07:20:29.122 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.130 INFO:teuthology.orchestra.run.smithi046.stdout:2907 writesame 0x244c1f5 thru 0x244d5ce data_size 0x2a(0x13da bytes) 2024-09-12T07:20:29.138 INFO:teuthology.orchestra.run.smithi046.stdout:2908 write 0x50ba62c thru 0x50ca5d9 (0xffae bytes) 2024-09-12T07:20:29.149 INFO:teuthology.orchestra.run.smithi046.stdout:2912 writesame 0x337c30f thru 0x337fcf7 data_size 0x5(0x39e9 bytes) 2024-09-12T07:20:29.153 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:29.154+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:2913 writesame 0xe1b002e thru 0xe1bd097 data_size 0x67(0xd06a bytes) 2024-09-12T07:20:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:2914 read 0x3276a58 thru 0x327c8fb (0x5ea4 bytes) 2024-09-12T07:20:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:2916 write 0xa0b31fa thru 0xa0bbe86 (0x8c8d bytes) 2024-09-12T07:20:29.169 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:29.169 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:29.176 INFO:teuthology.orchestra.run.smithi046.stdout:2917 read 0x4514878 thru 0x452443f (0xfbc8 bytes) 2024-09-12T07:20:29.176 INFO:teuthology.orchestra.run.smithi046.stdout:2918 read 0x4120c03 thru 0x412370c (0x2b0a bytes) 2024-09-12T07:20:29.176 INFO:teuthology.orchestra.run.smithi046.stdout:2919 writesame 0xc308c22 thru 0xc31463a data_size 0xb(0xba19 bytes) 2024-09-12T07:20:29.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:29.184 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.193 INFO:teuthology.orchestra.run.smithi046.stdout:2922 trunc from 0xe1bd098 to 0x6f2dc70 2024-09-12T07:20:29.204 INFO:teuthology.orchestra.run.smithi046.stdout:2923 read 0x6b1fabd thru 0x6b2ac24 (0xb168 bytes) 2024-09-12T07:20:29.206 INFO:teuthology.orchestra.run.smithi046.stdout:2924 trunc from 0x6f2dc70 to 0x8eef5ab 2024-09-12T07:20:29.212 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.220 INFO:teuthology.orchestra.run.smithi046.stdout:2928 write 0xb4babe6 thru 0xb4c91d5 (0xe5f0 bytes) 2024-09-12T07:20:29.230 INFO:teuthology.orchestra.run.smithi046.stdout:2929 punch from 0x126cfc8 to 0x1274b68, (0x7ba0 bytes) 2024-09-12T07:20:29.230 INFO:teuthology.orchestra.run.smithi046.stdout:2930 read 0x6296d3f thru 0x62a10a7 (0xa369 bytes) 2024-09-12T07:20:29.230 INFO:teuthology.orchestra.run.smithi046.stdout:2931 write 0x1343554 thru 0x134ef57 (0xba04 bytes) 2024-09-12T07:20:29.240 INFO:teuthology.orchestra.run.smithi046.stdout:2933 read 0x98960b9 thru 0x989ca14 (0x695c bytes) 2024-09-12T07:20:29.240 INFO:teuthology.orchestra.run.smithi046.stdout:2934 punch from 0xb0b4e57 to 0xb0b6597, (0x1740 bytes) 2024-09-12T07:20:29.240 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.240 INFO:teuthology.orchestra.run.smithi046.stdout:2937 write 0xa04f4d0 thru 0xa05207e (0x2baf bytes) 2024-09-12T07:20:29.244 INFO:teuthology.orchestra.run.smithi046.stdout:2938 read 0x466601b thru 0x4674660 (0xe646 bytes) 2024-09-12T07:20:29.248 INFO:teuthology.orchestra.run.smithi046.stdout:2943 trunc from 0xb4c91d6 to 0xe1d2b9f 2024-09-12T07:20:29.253 INFO:teuthology.orchestra.run.smithi046.stdout:2944 punch from 0x570d757 to 0x571aeab, (0xd754 bytes) 2024-09-12T07:20:29.253 INFO:teuthology.orchestra.run.smithi046.stdout:2948 read 0x16a7d57 thru 0x16aa930 (0x2bda bytes) 2024-09-12T07:20:29.254 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:29.255+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:29.254 INFO:teuthology.orchestra.run.smithi046.stdout:2949 write 0xf5aa45 thru 0xf6539a (0xa956 bytes) 2024-09-12T07:20:29.259 INFO:teuthology.orchestra.run.smithi046.stdout:2950 read 0xa31b282 thru 0xa321ac7 (0x6846 bytes) 2024-09-12T07:20:29.259 INFO:teuthology.orchestra.run.smithi046.stdout:2951 writesame 0xac8a31d thru 0xac99fa4 data_size 0x4(0xfc88 bytes) 2024-09-12T07:20:29.273 INFO:teuthology.orchestra.run.smithi046.stdout:2953 write 0x22bd005 thru 0x22c8142 (0xb13e bytes) 2024-09-12T07:20:29.282 INFO:teuthology.orchestra.run.smithi046.stdout:2955 read 0xb426b14 thru 0xb42ca14 (0x5f01 bytes) 2024-09-12T07:20:29.283 INFO:teuthology.orchestra.run.smithi046.stdout:2956 punch from 0x7a06646 to 0x7a13323, (0xccdd bytes) 2024-09-12T07:20:29.283 INFO:teuthology.orchestra.run.smithi046.stdout:2957 read 0x36c99af thru 0x36cda38 (0x408a bytes) 2024-09-12T07:20:29.283 INFO:teuthology.orchestra.run.smithi046.stdout:2958 write 0x412d231 thru 0x4138faf (0xbd7f bytes) 2024-09-12T07:20:29.288 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.2 to 1 (10000) 2024-09-12T07:20:29.292 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:20:29.299 INFO:teuthology.orchestra.run.smithi046.stdout:2959 write 0x5e162e5 thru 0x5e1de20 (0x7b3c bytes) 2024-09-12T07:20:29.306 INFO:teuthology.orchestra.run.smithi046.stdout:2962 punch from 0x258cea to 0x265662, (0xc978 bytes) 2024-09-12T07:20:29.306 INFO:teuthology.orchestra.run.smithi046.stdout:2963 write 0x88ffc05 thru 0x8900cf7 (0x10f3 bytes) 2024-09-12T07:20:29.310 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-12T07:20:29.316 INFO:teuthology.orchestra.run.smithi046.stdout:2967 write 0xe4ae5d3 thru 0xe4bce5c (0xe88a bytes) 2024-09-12T07:20:29.327 INFO:teuthology.orchestra.run.smithi046.stdout:2968 write 0xaebcae1 thru 0xaec0ded (0x430d bytes) 2024-09-12T07:20:29.329 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:29.329 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:29.330 INFO:teuthology.orchestra.run.smithi046.stdout:2969 punch from 0xcfafec7 to 0xcfb2e41, (0x2f7a bytes) 2024-09-12T07:20:29.330 INFO:teuthology.orchestra.run.smithi046.stdout:2970 read 0xb762f5a thru 0xb76bb83 (0x8c2a bytes) 2024-09-12T07:20:29.330 INFO:teuthology.orchestra.run.smithi046.stdout:2971 read 0x29462da thru 0x294b5b0 (0x52d7 bytes) 2024-09-12T07:20:29.330 INFO:teuthology.orchestra.run.smithi046.stdout:2972 read 0x8b239f thru 0x8b5b8d (0x37ef bytes) 2024-09-12T07:20:29.331 INFO:teuthology.orchestra.run.smithi046.stdout:2973 punch from 0x240764c to 0x240c45c, (0x4e10 bytes) 2024-09-12T07:20:29.331 INFO:teuthology.orchestra.run.smithi046.stdout:2975 read 0x850d7bc thru 0x851c4ea (0xed2f bytes) 2024-09-12T07:20:29.331 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.331 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.331 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.332 INFO:teuthology.orchestra.run.smithi046.stdout:2981 writesame 0x42f6ab0 thru 0x42f9b23 data_size 0xe(0x3074 bytes) 2024-09-12T07:20:29.333 INFO:teuthology.orchestra.run.smithi046.stdout:2982 read 0xcab0b9c thru 0xcab3519 (0x297e bytes) 2024-09-12T07:20:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:2985 trunc from 0xe4bce5d to 0x80ff43e 2024-09-12T07:20:29.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:29.355 INFO:teuthology.orchestra.run.smithi046.stdout:2986 punch from 0x4954fe7 to 0x495a709, (0x5722 bytes) 2024-09-12T07:20:29.355 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:29.357+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:29.363 INFO:teuthology.orchestra.run.smithi046.stdout:2987 write 0xa8cf154 thru 0xa8d1052 (0x1eff bytes) 2024-09-12T07:20:29.368 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.368 INFO:teuthology.orchestra.run.smithi046.stdout:2989 punch from 0x9e2489e to 0x9e31b51, (0xd2b3 bytes) 2024-09-12T07:20:29.379 INFO:teuthology.orchestra.run.smithi046.stdout:2990 write 0xe7a6cfd thru 0xe7ad19f (0x64a3 bytes) 2024-09-12T07:20:29.386 INFO:teuthology.orchestra.run.smithi046.stdout:2992 read 0x5c6867f thru 0x5c7169c (0x901e bytes) 2024-09-12T07:20:29.386 INFO:teuthology.orchestra.run.smithi046.stdout:2994 punch from 0xe2c5164 to 0xe2c5acb, (0x967 bytes) 2024-09-12T07:20:29.386 INFO:teuthology.orchestra.run.smithi046.stdout:2995 write 0x44d6aea thru 0x44e0997 (0x9eae bytes) 2024-09-12T07:20:29.390 INFO:teuthology.orchestra.run.smithi046.stdout:2997 read 0xe4761fe thru 0xe47bef4 (0x5cf7 bytes) 2024-09-12T07:20:29.391 INFO:teuthology.orchestra.run.smithi046.stdout:2999 read 0xba884de thru 0xba8a084 (0x1ba7 bytes) 2024-09-12T07:20:29.391 INFO:teuthology.orchestra.run.smithi046.stdout:3002 write 0x13010e3 thru 0x1303db9 (0x2cd7 bytes) 2024-09-12T07:20:29.392 INFO:teuthology.orchestra.run.smithi046.stdout:3004 read 0x224245f thru 0x22516ed (0xf28f bytes) 2024-09-12T07:20:29.396 INFO:teuthology.orchestra.run.smithi046.stdout:3005 trunc from 0xe7ad1a0 to 0xc1363b7 2024-09-12T07:20:29.407 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.407 INFO:teuthology.orchestra.run.smithi046.stdout:3007 writesame 0x5dd396c thru 0x5ddedc9 data_size 0x1(0xb45e bytes) 2024-09-12T07:20:29.431 INFO:teuthology.orchestra.run.smithi046.stdout:3008 trunc from 0xc1363b7 to 0x53a645c 2024-09-12T07:20:29.456 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:29.457+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:29.457 INFO:teuthology.orchestra.run.smithi046.stdout:3010 trunc from 0x53a645c to 0xe41a81d 2024-09-12T07:20:29.463 INFO:teuthology.orchestra.run.smithi046.stdout:3012 write 0xce97d8 thru 0xcee0ca (0x48f3 bytes) 2024-09-12T07:20:29.464 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.464 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.465 INFO:teuthology.orchestra.run.smithi046.stdout:3015 trunc from 0xe41a81d to 0xe19775a 2024-09-12T07:20:29.470 INFO:teuthology.orchestra.run.smithi046.stdout:3016 write 0x10f2004 thru 0x10fe155 (0xc152 bytes) 2024-09-12T07:20:29.474 INFO:teuthology.orchestra.run.smithi046.stdout:3017 write 0x9858df1 thru 0x98685fc (0xf80c bytes) 2024-09-12T07:20:29.486 INFO:teuthology.orchestra.run.smithi046.stdout:3019 write 0x5a9fc77 thru 0x5aa9de9 (0xa173 bytes) 2024-09-12T07:20:29.495 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:29.495 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:29.496 INFO:teuthology.orchestra.run.smithi046.stdout:3020 writesame 0x301766d thru 0x301861f data_size 0x1(0xfb3 bytes) 2024-09-12T07:20:29.501 INFO:teuthology.orchestra.run.smithi046.stdout:3023 read 0x9f3b578 thru 0x9f3f75f (0x41e8 bytes) 2024-09-12T07:20:29.501 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.501 INFO:teuthology.orchestra.run.smithi046.stdout:3025 punch from 0x3e77151 to 0x3e79660, (0x250f bytes) 2024-09-12T07:20:29.507 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:29.513 INFO:teuthology.orchestra.run.smithi046.stdout:3026 trunc from 0xe19775a to 0x386adb9 2024-09-12T07:20:29.531 INFO:teuthology.orchestra.run.smithi046.stdout:3028 read 0x35eb870 thru 0x35fb641 (0xfdd2 bytes) 2024-09-12T07:20:29.542 INFO:teuthology.orchestra.run.smithi046.stdout:3029 write 0x786605b thru 0x7872f27 (0xcecd bytes) 2024-09-12T07:20:29.552 INFO:teuthology.orchestra.run.smithi046.stdout:3030 read 0x6f3981d thru 0x6f45fa4 (0xc788 bytes) 2024-09-12T07:20:29.557 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:29.558+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:29.559 INFO:teuthology.orchestra.run.smithi046.stdout:3031 trunc from 0x7872f28 to 0xde336d9 2024-09-12T07:20:29.565 INFO:teuthology.orchestra.run.smithi046.stdout:3032 read 0x75b0001 thru 0x75b2687 (0x2687 bytes) 2024-09-12T07:20:29.565 INFO:teuthology.orchestra.run.smithi046.stdout:3035 read 0xc9056a3 thru 0xc907164 (0x1ac2 bytes) 2024-09-12T07:20:29.565 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.565 INFO:teuthology.orchestra.run.smithi046.stdout:3037 read 0xcdfcfe thru 0xce4743 (0x4a46 bytes) 2024-09-12T07:20:29.566 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.566 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.566 INFO:teuthology.orchestra.run.smithi046.stdout:3044 read 0x524b168 thru 0x52532a9 (0x8142 bytes) 2024-09-12T07:20:29.566 INFO:teuthology.orchestra.run.smithi046.stdout:3045 punch from 0x35ba361 to 0x35befca, (0x4c69 bytes) 2024-09-12T07:20:29.566 INFO:teuthology.orchestra.run.smithi046.stdout:3046 read 0xb8087c0 thru 0xb80a7f2 (0x2033 bytes) 2024-09-12T07:20:29.583 INFO:teuthology.orchestra.run.smithi046.stdout:3047 trunc from 0xde336d9 to 0x864cd4 2024-09-12T07:20:29.620 INFO:teuthology.orchestra.run.smithi046.stdout:3048 writesame 0x96a17dc thru 0x96b112e data_size 0x1(0xf953 bytes) 2024-09-12T07:20:29.620 INFO:teuthology.orchestra.run.smithi046.stdout:3050 read 0x4fecf62 thru 0x4ff22f7 (0x5396 bytes) 2024-09-12T07:20:29.620 INFO:teuthology.orchestra.run.smithi046.stdout:3051 writesame 0x54b7192 thru 0x54ba5fa data_size 0x1(0x3469 bytes) 2024-09-12T07:20:29.628 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:29.628 INFO:teuthology.orchestra.run.smithi046.stdout:3053 read 0x27c25d7 thru 0x27c7806 (0x5230 bytes) 2024-09-12T07:20:29.628 INFO:teuthology.orchestra.run.smithi046.stdout:3054 write 0x3194d42 thru 0x3199641 (0x4900 bytes) 2024-09-12T07:20:29.634 INFO:teuthology.orchestra.run.smithi046.stdout:3055 read 0x3870bd8 thru 0x38802eb (0xf714 bytes) 2024-09-12T07:20:29.634 INFO:teuthology.orchestra.run.smithi046.stdout:3057 write 0x7791c0c thru 0x779c3b5 (0xa7aa bytes) 2024-09-12T07:20:29.643 INFO:teuthology.orchestra.run.smithi046.stdout:3058 read 0x3afa366 thru 0x3b0138b (0x7026 bytes) 2024-09-12T07:20:29.650 INFO:teuthology.orchestra.run.smithi046.stdout:3059 trunc from 0x96b112f to 0xe48e95b 2024-09-12T07:20:29.657 INFO:teuthology.orchestra.run.smithi046.stdout:3060 read 0xcdb6925 thru 0xcdc0b3b (0xa217 bytes) 2024-09-12T07:20:29.657 INFO:teuthology.orchestra.run.smithi046.stdout:3061 read 0x5881092 thru 0x5890a0a (0xf979 bytes) 2024-09-12T07:20:29.657 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:29.659+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:29.657 INFO:teuthology.orchestra.run.smithi046.stdout:3063 write 0x44224f9 thru 0x4430a11 (0xe519 bytes) 2024-09-12T07:20:29.664 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:29.664 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:29.670 INFO:teuthology.orchestra.run.smithi046.stdout:3064 trunc from 0xe48e95b to 0xc2187fc 2024-09-12T07:20:29.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:29.677 INFO:teuthology.orchestra.run.smithi046.stdout:3065 read 0x8e53e75 thru 0x8e5a5bf (0x674b bytes) 2024-09-12T07:20:29.677 INFO:teuthology.orchestra.run.smithi046.stdout:3068 write 0x954f45c thru 0x95576e3 (0x8288 bytes) 2024-09-12T07:20:29.685 INFO:teuthology.orchestra.run.smithi046.stdout:3069 trunc from 0xc2187fc to 0xcc7b65b 2024-09-12T07:20:29.690 INFO:teuthology.orchestra.run.smithi046.stdout:3070 read 0xb0f619d thru 0xb0fa45d (0x42c1 bytes) 2024-09-12T07:20:29.690 INFO:teuthology.orchestra.run.smithi046.stdout:3071 clone 24 order 25 su 8388608 sc 8 2024-09-12T07:20:29.758 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:29.759+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:29.793 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:29.793 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:29.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:29.885 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:29.887+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:29.950 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:29.950 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:29.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:29.959 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:29.961+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:30.060 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:30.062+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:30.106 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:30.106 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:30.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:30.162 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:30.163+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:30.222 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:30.222 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:30.230 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:30.263 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:30.265+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.4 to 1 (10000) 2024-09-12T07:20:30.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-12T07:20:30.313 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:30.364 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:30.366+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:30.392 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:30.392 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:30.402 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:30.487 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:30.489+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:30.491 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:30.509 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:30.510 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:30.517 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-12T07:20:30.540 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:30.540 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:30.565 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:30.565 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:30.565 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:30.567+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:30.581 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77440 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13156352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.336686458999999, 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051399724000000001, 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77440 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246267+0000", 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082256199999999999 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246271+0000", 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6700000000000002e-06 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246283+0000", 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.207e-05 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246291+0000", 2024-09-12T07:20:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296645+0000", 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050353872000000001 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296729+0000", 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4215000000000006e-05 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296844+0000", 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114626 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77446 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.335687944, 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051543643, 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77446 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246579+0000", 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013651399999999999 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246583+0000", 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1509999999999997e-06 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246593+0000", 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5670000000000001e-06 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246599+0000", 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5189999999999998e-06 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297826+0000", 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051226381000000001 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297903+0000", 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6710000000000002e-05 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297986+0000", 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3801000000000006e-05 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77447 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 5783552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.335520330000001, 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051512637, 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77447 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246812+0000", 2024-09-12T07:20:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000201506 2024-09-12T07:20:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246817+0000", 2024-09-12T07:20:30.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:30.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246828+0000", 2024-09-12T07:20:30.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1693e-05 2024-09-12T07:20:30.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246836+0000", 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4429999999999997e-06 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298021+0000", 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051184940999999998 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298067+0000", 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6292000000000003e-05 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298123+0000", 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5828999999999997e-05 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77448 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.335280592, 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051506024999999997, 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.595 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77448 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246892+0000", 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1854999999999998e-05 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246895+0000", 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3129999999999999e-06 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246904+0000", 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6179999999999995e-06 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246911+0000", 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7939999999999997e-06 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298164+0000", 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051253559999999997 2024-09-12T07:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298249+0000", 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4549000000000007e-05 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298356+0000", 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107336 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.599 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77449 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.600 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:30.600 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.335209396, 2024-09-12T07:20:30.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051582830000000003, 2024-09-12T07:20:30.600 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77449 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247137+0000", 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021599499999999999 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247142+0000", 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247154+0000", 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1878e-05 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247168+0000", 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4114000000000001e-05 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298391+0000", 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051223249999999998 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298446+0000", 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4366000000000001e-05 2024-09-12T07:20:30.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298504+0000", 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8273000000000003e-05 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77450 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12681216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.334949825999999, 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051562375000000001, 2024-09-12T07:20:30.604 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77450 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2799999999999996e-07 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247184+0000", 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-12T07:20:30.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247192+0000", 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5920000000000003e-06 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247197+0000", 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6720000000000001e-06 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298550+0000", 2024-09-12T07:20:30.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051352082 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298641+0000", 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1236000000000005e-05 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298743+0000", 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102428 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77451 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7061504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.334925712, 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051813773, 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77451 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.609 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247335+0000", 2024-09-12T07:20:30.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013012599999999999 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247340+0000", 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.544e-06 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247351+0000", 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.111e-05 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247358+0000", 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9859999999999997e-06 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298909+0000", 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051551781999999997 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298960+0000", 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0594999999999999e-05 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299019+0000", 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8629999999999999e-05 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77452 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.334760932000002, 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051892134999999999, 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.613 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77452 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247535+0000", 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165508 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247552+0000", 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6514000000000001e-05 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247565+0000", 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3372e-05 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247577+0000", 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299064+0000", 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051486697999999997 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299150+0000", 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6549000000000001e-05 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299262+0000", 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011135 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.617 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77453 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.334540211, 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051901623000000001, 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77453 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247710+0000", 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011951 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247713+0000", 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4539999999999999e-06 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247720+0000", 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0339999999999999e-06 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247730+0000", 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0030000000000001e-05 2024-09-12T07:20:30.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299297+0000", 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051566446000000002 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299382+0000", 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4963000000000006e-05 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299492+0000", 2024-09-12T07:20:30.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011018600000000001 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77454 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7618560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.334391664000002, 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051921851999999998, 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77454 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247900+0000", 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016088699999999999 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247904+0000", 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6950000000000002e-06 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247912+0000", 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7129999999999994e-06 2024-09-12T07:20:30.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247921+0000", 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0299999999999999e-06 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299526+0000", 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051604744000000001 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299601+0000", 2024-09-12T07:20:30.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5129000000000005e-05 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299661+0000", 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9654000000000002e-05 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77455 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12746752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.334197796000002, 2024-09-12T07:20:30.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051971254000000001, 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77455 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:30.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248181+0000", 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024790000000000001 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248185+0000", 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5079999999999999e-06 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248197+0000", 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.132e-05 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248203+0000", 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6320000000000002e-06 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299705+0000", 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051501342999999998 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299799+0000", 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4340999999999997e-05 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299904+0000", 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010521 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.630 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77456 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7979008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.333915570999999, 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051860577999999997, 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77456 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248545+0000", 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033009700000000001 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248550+0000", 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248560+0000", 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248568+0000", 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4939999999999997e-06 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299973+0000", 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051405470000000002 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300019+0000", 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5448999999999998e-05 2024-09-12T07:20:30.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300076+0000", 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7073000000000001e-05 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77457 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13172736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.333549243, 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051737914000000003, 2024-09-12T07:20:30.635 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77457 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248582+0000", 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248584+0000", 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:20:30.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248593+0000", 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248609+0000", 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5664000000000001e-05 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300117+0000", 2024-09-12T07:20:30.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051507605999999997 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300212+0000", 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4919000000000004e-05 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300319+0000", 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010776399999999999 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77458 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.333506825000001, 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051863578, 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77458 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.640 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:30.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248627+0000", 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1389999999999999e-06 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248635+0000", 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4019999999999996e-06 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248641+0000", 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1689999999999997e-06 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300359+0000", 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051717889000000003 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300418+0000", 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8689e-05 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300487+0000", 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9756000000000001e-05 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77459 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13238272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.333481131999999, 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052076504000000003, 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.644 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77459 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250611+0000", 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019617879999999999 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250618+0000", 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1829999999999997e-06 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250629+0000", 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0438000000000001e-05 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250648+0000", 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9355e-05 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300528+0000", 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049879252999999998 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300619+0000", 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1910000000000006e-05 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300726+0000", 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010657700000000001 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.648 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77467 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.331275312999999, 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05160754, 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77467 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250856+0000", 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250858+0000", 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9180000000000001e-06 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250864+0000", 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.066e-06 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250869+0000", 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302283+0000", 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051414591000000003 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302378+0000", 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4283000000000005e-05 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302463+0000", 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5249999999999999e-05 2024-09-12T07:20:30.652 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77468 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14909440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.33125574, 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051742162000000001, 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.653 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77468 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250877+0000", 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.815e-06 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250883+0000", 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:20:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250888+0000", 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302496+0000", 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051607394000000001 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302553+0000", 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6971000000000001e-05 2024-09-12T07:20:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302617+0000", 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4418999999999994e-05 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77469 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14974976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.331235561, 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965694, 2024-09-12T07:20:30.657 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77469 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250897+0000", 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9130000000000001e-06 2024-09-12T07:20:30.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250903+0000", 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0129999999999999e-06 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250908+0000", 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9679999999999997e-06 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302661+0000", 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051753066 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302755+0000", 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3376999999999997e-05 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302861+0000", 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105924 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.661 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77470 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12828672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.331215883999999, 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052098589000000001, 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77470 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.662 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:30.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250917+0000", 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8190000000000001e-06 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250923+0000", 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250928+0000", 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9420000000000003e-06 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302894+0000", 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965863000000001 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302955+0000", 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1235999999999994e-05 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303013+0000", 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8167000000000001e-05 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77471 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12894208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.331196338000002, 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052326263999999997, 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:30.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77471 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:30.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256961+0000", 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060271209999999999 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256972+0000", 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0755e-05 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256986+0000", 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845e-05 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256996+0000", 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303060+0000", 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.046063661999999998 2024-09-12T07:20:30.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303149+0000", 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8949999999999994e-05 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303261+0000", 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011195 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:30.670 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:30.671 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:30.671 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:30.672 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:30.673 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:30.673 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:30.674 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:30.674 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:30.675 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:30.676 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:30.673+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:30.677 INFO:teuthology.orchestra.run.smithi046.stdout:3072 trunc from 0xcc7b65b to 0xa543bf6 2024-09-12T07:20:30.718 INFO:teuthology.orchestra.run.smithi046.stdout:3073 trunc from 0xa543bf6 to 0x6c56bce 2024-09-12T07:20:30.730 INFO:teuthology.orchestra.run.smithi046.stdout:3074 read 0x888acd thru 0x893499 (0xa9cd bytes) 2024-09-12T07:20:30.737 INFO:teuthology.orchestra.run.smithi046.stdout:3075 writesame 0x767b473 thru 0x7684bee data_size 0x23(0x977c bytes) 2024-09-12T07:20:30.738 INFO:teuthology.orchestra.run.smithi046.stdout:3076 read 0x15fa684 thru 0x15fe4c5 (0x3e42 bytes) 2024-09-12T07:20:30.738 INFO:teuthology.orchestra.run.smithi046.stdout:3077 punch from 0xef17e to 0xf6388, (0x720a bytes) 2024-09-12T07:20:30.738 INFO:teuthology.orchestra.run.smithi046.stdout:3079 writesame 0x565aeb8 thru 0x5663db3 data_size 0x2(0x8efc bytes) 2024-09-12T07:20:30.751 INFO:teuthology.orchestra.run.smithi046.stdout:3081 trunc from 0x7684bef to 0x3bb83a1 2024-09-12T07:20:30.758 INFO:teuthology.orchestra.run.smithi046.stdout:3082 read 0x1c0bc1b thru 0x1c14f93 (0x9379 bytes) 2024-09-12T07:20:30.762 INFO:teuthology.orchestra.run.smithi046.stdout:3083 trunc from 0x3bb83a1 to 0xca3b75 2024-09-12T07:20:30.772 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:30.774+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:30.782 INFO:teuthology.orchestra.run.smithi046.stdout:3084 trunc from 0xca3b75 to 0x36e5743 2024-09-12T07:20:30.787 INFO:teuthology.orchestra.run.smithi046.stdout:3085 read 0x2812d4b thru 0x281c634 (0x98ea bytes) 2024-09-12T07:20:30.787 INFO:teuthology.orchestra.run.smithi046.stdout:3086 read 0x13678a8 thru 0x137329e (0xb9f7 bytes) 2024-09-12T07:20:30.787 INFO:teuthology.orchestra.run.smithi046.stdout:3087 read 0x2de2809 thru 0x2de2def (0x5e7 bytes) 2024-09-12T07:20:30.794 INFO:teuthology.orchestra.run.smithi046.stdout:3088 writesame 0x4dd3d30 thru 0x4dd43f0 data_size 0xd(0x6c1 bytes) 2024-09-12T07:20:30.807 INFO:teuthology.orchestra.run.smithi046.stdout:3090 write 0x9ff19ad thru 0x9ffa3db (0x8a2f bytes) 2024-09-12T07:20:30.823 INFO:teuthology.orchestra.run.smithi046.stdout:3094 trunc from 0x9ffa3dc to 0x4d2d10 2024-09-12T07:20:30.833 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:30.833 INFO:teuthology.orchestra.run.smithi046.stdout:3097 read 0x5fd40 thru 0x6f47b (0xf73c bytes) 2024-09-12T07:20:30.836 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:30.836 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:30.837 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:30.837 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:30.846 INFO:teuthology.orchestra.run.smithi046.stdout:3098 write 0x5fb805a thru 0x5fbf5a3 (0x754a bytes) 2024-09-12T07:20:30.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:30.860 INFO:teuthology.orchestra.run.smithi046.stdout:3099 writesame 0xb5b6e69 thru 0xb5c28c4 data_size 0x2(0xba5c bytes) 2024-09-12T07:20:30.863 INFO:teuthology.orchestra.run.smithi046.stdout:3100 trunc from 0xb5c28c5 to 0x8de8a80 2024-09-12T07:20:30.873 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:30.875+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:30.879 INFO:teuthology.orchestra.run.smithi046.stdout:3101 trunc from 0x8de8a80 to 0x2b12bf 2024-09-12T07:20:30.888 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:30.888 INFO:teuthology.orchestra.run.smithi046.stdout:3104 punch from 0xd3883 to 0xd9c26, (0x63a3 bytes) 2024-09-12T07:20:30.889 INFO:teuthology.orchestra.run.smithi046.stdout:3105 read 0x10c18b thru 0x1181df (0xc055 bytes) 2024-09-12T07:20:30.889 INFO:teuthology.orchestra.run.smithi046.stdout:3106 read 0x237d35 thru 0x23e9a8 (0x6c74 bytes) 2024-09-12T07:20:30.889 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:30.904 INFO:teuthology.orchestra.run.smithi046.stdout:3109 trunc from 0x2b12bf to 0xd3d40e7 2024-09-12T07:20:30.913 INFO:teuthology.orchestra.run.smithi046.stdout:3110 read 0xc4a4003 thru 0xc4a725b (0x3259 bytes) 2024-09-12T07:20:30.913 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:30.913 INFO:teuthology.orchestra.run.smithi046.stdout:3112 punch from 0x5b25727 to 0x5b26507, (0xde0 bytes) 2024-09-12T07:20:30.913 INFO:teuthology.orchestra.run.smithi046.stdout:3113 write 0x731c6f5 thru 0x7324fed (0x88f9 bytes) 2024-09-12T07:20:30.914 INFO:teuthology.orchestra.run.smithi046.stdout:3114 writesame 0x2d64e28 thru 0x2d7108a data_size 0x1(0xc263 bytes) 2024-09-12T07:20:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:3115 write 0xb367a19 thru 0xb375286 (0xd86e bytes) 2024-09-12T07:20:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:3116 trunc from 0xd3d40e7 to 0x884af86 2024-09-12T07:20:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:3117 punch from 0x6f89c8c to 0x6f98d3c, (0xf0b0 bytes) 2024-09-12T07:20:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:3118 trunc from 0x884af86 to 0x6f23af 2024-09-12T07:20:30.974 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:30.976+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:3120 punch from 0xe6384 to 0xf232e, (0xbfaa bytes) 2024-09-12T07:20:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:30.983 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:30.984 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:30.984 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:3121 trunc from 0x6f23af to 0xace2863 2024-09-12T07:20:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:3122 writesame 0x8d7fdef thru 0x8d823ac data_size 0x1(0x25be bytes) 2024-09-12T07:20:30.996 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:3123 writesame 0xd171c9e thru 0xd17e97c data_size 0x1(0xccdf bytes) 2024-09-12T07:20:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:3126 write 0xf81015 thru 0xf8c5b4 (0xb5a0 bytes) 2024-09-12T07:20:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:3128 write 0x48cda9d thru 0x48d921e (0xb782 bytes) 2024-09-12T07:20:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:3129 writesame 0x155982e thru 0x1561bd0 data_size 0x3(0x83a3 bytes) 2024-09-12T07:20:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:3130 trunc from 0xd17e97d to 0x82201b8 2024-09-12T07:20:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:3131 writesame 0x6824815 thru 0x682de6e data_size 0xf(0x965a bytes) 2024-09-12T07:20:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:3132 write 0x5cfbae0 thru 0x5d072f5 (0xb816 bytes) 2024-09-12T07:20:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:3133 writesame 0xe312cc5 thru 0xe3213d9 data_size 0x15(0xe715 bytes) 2024-09-12T07:20:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:3135 read 0x4de5f17 thru 0x4de9a0d (0x3af7 bytes) 2024-09-12T07:20:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:3136 writesame 0xedfd4ce thru 0xee02b68 data_size 0x1(0x569b bytes) 2024-09-12T07:20:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:3138 writesame 0xd39a45d thru 0xd39f878 data_size 0xe(0x541c bytes) 2024-09-12T07:20:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:3140 punch from 0xc1e50c1 to 0xc1ed186, (0x80c5 bytes) 2024-09-12T07:20:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:3142 write 0x948b96c thru 0x9493df9 (0x848e bytes) 2024-09-12T07:20:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:3143 punch from 0xd5e6b80 to 0xd5f3074, (0xc4f4 bytes) 2024-09-12T07:20:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:3144 write 0xe408cc2 thru 0xe416ee2 (0xe221 bytes) 2024-09-12T07:20:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:3145 write 0xb4f43f4 thru 0xb5024c5 (0xe0d2 bytes) 2024-09-12T07:20:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:3146 write 0x926d929 thru 0x927cfb5 (0xf68d bytes) 2024-09-12T07:20:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:3147 read 0xdfa57 thru 0xecb34 (0xd0de bytes) 2024-09-12T07:20:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:3148 writesame 0x7019652 thru 0x7025569 data_size 0x14(0xbf18 bytes) 2024-09-12T07:20:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:3151 write 0x12b97e7 thru 0x12c83dc (0xebf6 bytes) 2024-09-12T07:20:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:3154 write 0x37d0a0a thru 0x37e0083 (0xf67a bytes) 2024-09-12T07:20:31.075 INFO:teuthology.orchestra.run.smithi046.stdout:3155 write 0x1fbea29 thru 0x1fc45d2 (0x5baa bytes) 2024-09-12T07:20:31.075 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:31.077+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:31.076 INFO:teuthology.orchestra.run.smithi046.stdout:3156 punch from 0xba11bc1 to 0xba13a34, (0x1e73 bytes) 2024-09-12T07:20:31.076 INFO:teuthology.orchestra.run.smithi046.stdout:3157 write 0x47e16a6 thru 0x47e336e (0x1cc9 bytes) 2024-09-12T07:20:31.077 INFO:teuthology.orchestra.run.smithi046.stdout:3159 punch from 0x8a704e2 to 0x8a7dd7a, (0xd898 bytes) 2024-09-12T07:20:31.078 INFO:teuthology.orchestra.run.smithi046.stdout:3160 write 0xc10f877 thru 0xc114cb9 (0x5443 bytes) 2024-09-12T07:20:31.085 INFO:teuthology.orchestra.run.smithi046.stdout:3162 trunc from 0xee02b69 to 0x921b97e 2024-09-12T07:20:31.095 INFO:teuthology.orchestra.run.smithi046.stdout:3163 trunc from 0x921b97e to 0xbae2474 2024-09-12T07:20:31.100 INFO:teuthology.orchestra.run.smithi046.stdout:3164 write 0x61ff0f3 thru 0x620079b (0x16a9 bytes) 2024-09-12T07:20:31.101 INFO:teuthology.orchestra.run.smithi046.stdout:3166 write 0xab62aa5 thru 0xab6be66 (0x93c2 bytes) 2024-09-12T07:20:31.103 INFO:teuthology.orchestra.run.smithi046.stdout:3167 write 0x2511325 thru 0x251d73c (0xc418 bytes) 2024-09-12T07:20:31.104 INFO:teuthology.orchestra.run.smithi046.stdout:3169 write 0xb60772d thru 0xb609d20 (0x25f4 bytes) 2024-09-12T07:20:31.105 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.106 INFO:teuthology.orchestra.run.smithi046.stdout:3172 read 0x8237c61 thru 0x8242c1d (0xafbd bytes) 2024-09-12T07:20:31.106 INFO:teuthology.orchestra.run.smithi046.stdout:3174 write 0xba3bf71 thru 0xba4827b (0xc30b bytes) 2024-09-12T07:20:31.110 INFO:teuthology.orchestra.run.smithi046.stdout:3175 write 0x7629410 thru 0x762a255 (0xe46 bytes) 2024-09-12T07:20:31.113 INFO:teuthology.orchestra.run.smithi046.stdout:3178 trunc from 0xbae2474 to 0x9ac7d30 2024-09-12T07:20:31.119 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.120 INFO:teuthology.orchestra.run.smithi046.stdout:3180 read 0x423d00b thru 0x423dc4a (0xc40 bytes) 2024-09-12T07:20:31.120 INFO:teuthology.orchestra.run.smithi046.stdout:3182 read 0x3e8f496 thru 0x3e8fde2 (0x94d bytes) 2024-09-12T07:20:31.121 INFO:teuthology.orchestra.run.smithi046.stdout:3185 read 0x9a57767 thru 0x9a64eab (0xd745 bytes) 2024-09-12T07:20:31.121 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.121 INFO:teuthology.orchestra.run.smithi046.stdout:3187 punch from 0x233f32d to 0x23495b5, (0xa288 bytes) 2024-09-12T07:20:31.126 INFO:teuthology.orchestra.run.smithi046.stdout:3188 write 0x9d2be46 thru 0x9d39fea (0xe1a5 bytes) 2024-09-12T07:20:31.137 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126196 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.6171880949999999, 2024-09-12T07:20:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038184672000000003, 2024-09-12T07:20:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126196 2024-09-12T07:20:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6199999999999999e-07 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521351+0000", 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9520000000000001e-06 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521357+0000", 2024-09-12T07:20:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-12T07:20:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521362+0000", 2024-09-12T07:20:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9180000000000002e-06 2024-09-12T07:20:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559327+0000", 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037964880999999999 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559401+0000", 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3750000000000004e-05 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559533+0000", 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013262300000000001 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126334 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.616607358, 2024-09-12T07:20:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038163516000000001, 2024-09-12T07:20:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126334 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521930+0000", 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521932+0000", 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9420000000000002e-06 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521938+0000", 2024-09-12T07:20:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.127e-06 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521949+0000", 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1538e-05 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559814+0000", 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037864291000000001 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559903+0000", 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9409999999999999e-05 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560093+0000", 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018984899999999999 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126336 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.616579121, 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038384952, 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126336 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2e-07 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521962+0000", 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.591e-06 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521969+0000", 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9199999999999998e-06 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521974+0000", 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2179999999999998e-06 2024-09-12T07:20:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560137+0000", 2024-09-12T07:20:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038163034999999998 2024-09-12T07:20:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560193+0000", 2024-09-12T07:20:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6425000000000003e-05 2024-09-12T07:20:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560343+0000", 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000149343 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126337 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.61655601, 2024-09-12T07:20:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038587955, 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126337 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521983+0000", 2024-09-12T07:20:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2079999999999999e-06 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521989+0000", 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8649999999999998e-06 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521994+0000", 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8720000000000001e-06 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560393+0000", 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038399232999999998 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560442+0000", 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8310999999999999e-05 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560569+0000", 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012708700000000001 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126339 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.6165361360000001, 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038809422000000003, 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126339 2024-09-12T07:20:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.58e-07 2024-09-12T07:20:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522003+0000", 2024-09-12T07:20:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0059999999999999e-06 2024-09-12T07:20:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522009+0000", 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522014+0000", 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0409999999999997e-06 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560605+0000", 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038590994000000003 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560650+0000", 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5336000000000001e-05 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560810+0000", 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000159853 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126341 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.616516807, 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039112779, 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126341 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6399999999999998e-07 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522022+0000", 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522028+0000", 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9089999999999997e-06 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522035+0000", 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5200000000000003e-06 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560857+0000", 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038822239000000001 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560955+0000", 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7503999999999993e-05 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561133+0000", 2024-09-12T07:20:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017827899999999999 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126342 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.616494871, 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039296258000000001, 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126342 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.84e-07 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522044+0000", 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522057+0000", 2024-09-12T07:20:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330999999999999e-05 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522062+0000", 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3700000000000003e-06 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561151+0000", 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039089193000000001 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561215+0000", 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3305000000000005e-05 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561338+0000", 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012353900000000001 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126343 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.6164667530000001, 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039573481000000001, 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126343 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522071+0000", 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2599999999999998e-07 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522077+0000", 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2049999999999999e-06 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522084+0000", 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1679999999999998e-06 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522089+0000", 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561375+0000", 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039286692999999998 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561463+0000", 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7575000000000001e-05 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561644+0000", 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018070799999999999 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126344 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.6164402369999999, 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039759638, 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126344 2024-09-12T07:20:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4900000000000001e-07 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522099+0000", 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4250000000000001e-06 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522106+0000", 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.455e-06 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522112+0000", 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3169999999999998e-06 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561676+0000", 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039564153999999997 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561728+0000", 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1848999999999998e-05 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561856+0000", 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012808900000000001 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126345 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.6164136440000001, 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040055773000000003, 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126345 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522124+0000", 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6600000000000002e-07 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522130+0000", 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.72e-06 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522140+0000", 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0217000000000001e-05 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522146+0000", 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3380000000000004e-06 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561896+0000", 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039750343 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561997+0000", 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010129899999999999 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562179+0000", 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018149 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126346 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.6162064090000001, 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040058279000000002, 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126346 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522331+0000", 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522334+0000", 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.01e-06 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522346+0000", 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522353+0000", 2024-09-12T07:20:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5590000000000002e-06 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562212+0000", 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039859232000000001 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562256+0000", 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4354999999999998e-05 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562389+0000", 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013237899999999999 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126348 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.6161737240000003, 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040352420999999999, 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126348 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522364+0000", 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.51e-07 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522366+0000", 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522373+0000", 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3339999999999996e-06 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522378+0000", 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8620000000000002e-06 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562428+0000", 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040049840000000003 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562522+0000", 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3589000000000001e-05 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562716+0000", 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019395799999999999 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126349 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1269760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.616150916, 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040535117000000002, 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126349 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522389+0000", 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.317e-06 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522396+0000", 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522401+0000", 2024-09-12T07:20:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5770000000000001e-06 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562754+0000", 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040352271000000002 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562809+0000", 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5376e-05 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562921+0000", 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112098 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126350 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.616128427, 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040836117999999998, 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126350 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522409+0000", 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9099999999999999e-07 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522411+0000", 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1449999999999998e-06 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522417+0000", 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1070000000000001e-06 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522422+0000", 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7600000000000002e-06 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562965+0000", 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040543467 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563058+0000", 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2905000000000006e-05 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563245+0000", 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186343 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126351 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.6159375860000003, 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040915973000000001, 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126351 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522600+0000", 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7899999999999998e-07 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522604+0000", 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-06 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522615+0000", 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1754e-05 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522622+0000", 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.798e-06 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563277+0000", 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040654765000000002 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563340+0000", 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2580000000000001e-05 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563515+0000", 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017560699999999999 2024-09-12T07:20:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126352 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.6159022919999999, 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041184365000000001, 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126352 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.99e-07 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522637+0000", 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3180000000000001e-06 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522645+0000", 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0309999999999998e-06 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522651+0000", 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.789e-06 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563559+0000", 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040907261 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563634+0000", 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5708000000000001e-05 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563819+0000", 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000184659 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126353 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.6158768390000002, 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388632000000002, 2024-09-12T07:20:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126353 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522661+0000", 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7299999999999996e-07 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522663+0000", 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522670+0000", 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6850000000000004e-06 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522675+0000", 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9139999999999999e-06 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563854+0000", 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041178851000000002 2024-09-12T07:20:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563912+0000", 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8584e-05 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564049+0000", 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000136427 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126354 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.6158543129999998, 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041618080000000002, 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126354 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2399999999999999e-07 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522685+0000", 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2469999999999998e-06 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522691+0000", 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.353e-06 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522697+0000", 2024-09-12T07:20:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.113e-06 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564085+0000", 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388104000000002 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564149+0000", 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4471e-05 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564301+0000", 2024-09-12T07:20:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015146800000000001 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:129861 3.18 3:1cf16ce6:::rbd_data.116031710b3d.00000000000000ae:18 [read 40960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.3659186019999998, 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038044815000000003, 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 129861 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772619+0000", 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5400000000000002e-07 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772620+0000", 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6980000000000001e-06 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772627+0000", 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3119999999999999e-06 2024-09-12T07:20:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772631+0000", 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9010000000000002e-06 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810471+0000", 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037839308000000002 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810521+0000", 2024-09-12T07:20:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9988999999999999e-05 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810663+0000", 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014225300000000001 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:129880 3.8 3:149a3dec:::rbd_data.116031710b3d.00000000000000ab:18 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.3658783859999999, 2024-09-12T07:20:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038109654999999999, 2024-09-12T07:20:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 129880 2024-09-12T07:20:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772659+0000", 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772661+0000", 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8279999999999999e-06 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772666+0000", 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6319999999999996e-06 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772672+0000", 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0510000000000002e-06 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810577+0000", 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037904895000000001 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810688+0000", 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011072400000000001 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810768+0000", 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0139e-05 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:31.228 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:31.229 INFO:teuthology.orchestra.run.smithi046.stdout:3190 write 0xe9ff130 thru 0xe9ffdc9 (0xc9a bytes) 2024-09-12T07:20:31.229 INFO:teuthology.orchestra.run.smithi046.stdout:3192 read 0x221076d thru 0x221214b (0x19df bytes) 2024-09-12T07:20:31.229 INFO:teuthology.orchestra.run.smithi046.stdout:3193 write 0xadb4b01 thru 0xadb9065 (0x4565 bytes) 2024-09-12T07:20:31.229 INFO:teuthology.orchestra.run.smithi046.stdout:3194 punch from 0xe035be6 to 0xe0386ff, (0x2b19 bytes) 2024-09-12T07:20:31.230 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:31.232 INFO:teuthology.orchestra.run.smithi046.stdout:3195 trunc from 0xe9ffdca to 0x3121e06 2024-09-12T07:20:31.232 INFO:teuthology.orchestra.run.smithi046.stdout:3197 read 0x1accc48 thru 0x1ad7f58 (0xb311 bytes) 2024-09-12T07:20:31.232 INFO:teuthology.orchestra.run.smithi046.stdout:3198 read 0x2143074 thru 0x21493f0 (0x637d bytes) 2024-09-12T07:20:31.232 INFO:teuthology.orchestra.run.smithi046.stdout:3200 read 0x26b22e thru 0x26e9d9 (0x37ac bytes) 2024-09-12T07:20:31.232 INFO:teuthology.orchestra.run.smithi046.stdout:3201 punch from 0x55b3d1 to 0x564e0a, (0x9a39 bytes) 2024-09-12T07:20:31.232 INFO:teuthology.orchestra.run.smithi046.stdout:3204 write 0x247597 thru 0x24a5e7 (0x3051 bytes) 2024-09-12T07:20:31.232 INFO:teuthology.orchestra.run.smithi046.stdout:3205 write 0x53e6696 thru 0x53f41fe (0xdb69 bytes) 2024-09-12T07:20:31.232 INFO:teuthology.orchestra.run.smithi046.stdout:3206 read 0x3ea5425 thru 0x3ea642b (0x1007 bytes) 2024-09-12T07:20:31.233 INFO:teuthology.orchestra.run.smithi046.stdout:3207 write 0x6e3a0ce thru 0x6e3c48b (0x23be bytes) 2024-09-12T07:20:31.233 INFO:teuthology.orchestra.run.smithi046.stdout:3208 writesame 0x449e94c thru 0x44a1271 data_size 0x2(0x2926 bytes) 2024-09-12T07:20:31.233 INFO:teuthology.orchestra.run.smithi046.stdout:3210 read 0x24f7f48 thru 0x24feac8 (0x6b81 bytes) 2024-09-12T07:20:31.233 INFO:teuthology.orchestra.run.smithi046.stdout:3215 write 0xd78971e thru 0xd79435a (0xac3d bytes) 2024-09-12T07:20:31.233 INFO:teuthology.orchestra.run.smithi046.stdout:3216 read 0x756aa3c thru 0x756ddf1 (0x33b6 bytes) 2024-09-12T07:20:31.233 INFO:teuthology.orchestra.run.smithi046.stdout:3219 read 0x23d6e7a thru 0x23dd5a2 (0x6729 bytes) 2024-09-12T07:20:31.233 INFO:teuthology.orchestra.run.smithi046.stdout:3220 punch from 0x4a71458 to 0x4a7b6b1, (0xa259 bytes) 2024-09-12T07:20:31.233 INFO:teuthology.orchestra.run.smithi046.stdout:3221 read 0x1121b5 thru 0x11249d (0x2e9 bytes) 2024-09-12T07:20:31.233 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.233 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.233 INFO:teuthology.orchestra.run.smithi046.stdout:3227 write 0x3ea06dd thru 0x3eacfbd (0xc8e1 bytes) 2024-09-12T07:20:31.233 INFO:teuthology.orchestra.run.smithi046.stdout:3229 trunc from 0xd79435b to 0xe9058e 2024-09-12T07:20:31.234 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:31.230+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:31.247 INFO:teuthology.orchestra.run.smithi046.stdout:3230 write 0x65db119 thru 0x65e2899 (0x7781 bytes) 2024-09-12T07:20:31.250 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.250 INFO:teuthology.orchestra.run.smithi046.stdout:3234 write 0x3194941 thru 0x3198e02 (0x44c2 bytes) 2024-09-12T07:20:31.253 INFO:teuthology.orchestra.run.smithi046.stdout:3236 read 0x20afb2 thru 0x21090f (0x595e bytes) 2024-09-12T07:20:31.254 INFO:teuthology.orchestra.run.smithi046.stdout:3239 write 0x49dee1f thru 0x49ee328 (0xf50a bytes) 2024-09-12T07:20:31.256 INFO:teuthology.orchestra.run.smithi046.stdout:3240 read 0x446d4ef thru 0x4470912 (0x3424 bytes) 2024-09-12T07:20:31.256 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.260 INFO:teuthology.orchestra.run.smithi046.stdout:3242 trunc from 0x65e289a to 0x3373b48 2024-09-12T07:20:31.266 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.281 INFO:teuthology.orchestra.run.smithi046.stdout:3244 writesame 0xc3b5455 thru 0xc3bba71 data_size 0x1(0x661d bytes) 2024-09-12T07:20:31.283 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.283 INFO:teuthology.orchestra.run.smithi046.stdout:3247 read 0x6888fb1 thru 0x68971fb (0xe24b bytes) 2024-09-12T07:20:31.290 INFO:teuthology.orchestra.run.smithi046.stdout:3248 writesame 0xd9917c8 thru 0xd998862 data_size 0x3(0x709b bytes) 2024-09-12T07:20:31.299 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.5 to 1 (10000) 2024-09-12T07:20:31.300 INFO:teuthology.orchestra.run.smithi046.stdout:3249 trunc from 0xd998863 to 0x504eb2b 2024-09-12T07:20:31.311 INFO:teuthology.orchestra.run.smithi046.stdout:3250 writesame 0x77af32 thru 0x787e8c data_size 0x1(0xcf5b bytes) 2024-09-12T07:20:31.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T07:20:31.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-12T07:20:31.330 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:31.331+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:31.344 INFO:teuthology.orchestra.run.smithi046.stdout:3252 trunc from 0x504eb2b to 0xb68274b 2024-09-12T07:20:31.345 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:31.346 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:31.346 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:31.346 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:31.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:31.359 INFO:teuthology.orchestra.run.smithi046.stdout:3253 writesame 0xd79910b thru 0xd7a53d0 data_size 0x2(0xc2c6 bytes) 2024-09-12T07:20:31.359 INFO:teuthology.orchestra.run.smithi046.stdout:3255 write 0x977f9e1 thru 0x97828b1 (0x2ed1 bytes) 2024-09-12T07:20:31.362 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.372 INFO:teuthology.orchestra.run.smithi046.stdout:3257 trunc from 0xd7a53d1 to 0x46716e9 2024-09-12T07:20:31.397 INFO:teuthology.orchestra.run.smithi046.stdout:3259 write 0xe34cd43 thru 0xe350c9e (0x3f5c bytes) 2024-09-12T07:20:31.408 INFO:teuthology.orchestra.run.smithi046.stdout:3260 trunc from 0xe350c9f to 0x517cebe 2024-09-12T07:20:31.416 INFO:teuthology.orchestra.run.smithi046.stdout:3261 punch from 0x1939800 to 0x1944596, (0xad96 bytes) 2024-09-12T07:20:31.416 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.430 INFO:teuthology.orchestra.run.smithi046.stdout:3264 write 0xbe2cbe6 thru 0xbe37801 (0xac1c bytes) 2024-09-12T07:20:31.433 INFO:teuthology.orchestra.run.smithi046.stdout:3265 read 0x1d5661c thru 0x1d59db4 (0x3799 bytes) 2024-09-12T07:20:31.433 INFO:teuthology.orchestra.run.smithi046.stdout:3267 read 0x5fcc8f1 thru 0x5fce16b (0x187b bytes) 2024-09-12T07:20:31.433 INFO:teuthology.orchestra.run.smithi046.stdout:3269 writesame 0x1736b81 thru 0x173e7c4 data_size 0x21(0x7c44 bytes) 2024-09-12T07:20:31.435 INFO:teuthology.orchestra.run.smithi046.stdout:3270 punch from 0x417a786 to 0x4186e39, (0xc6b3 bytes) 2024-09-12T07:20:31.435 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.435 INFO:teuthology.orchestra.run.smithi046.stdout:3274 punch from 0x73e526 to 0x73ee41, (0x91b bytes) 2024-09-12T07:20:31.435 INFO:teuthology.orchestra.run.smithi046.stdout:3275 read 0x940c86 thru 0x944407 (0x3782 bytes) 2024-09-12T07:20:31.438 INFO:teuthology.orchestra.run.smithi046.stdout:3276 trunc from 0xbe37802 to 0xe9d9195 2024-09-12T07:20:31.440 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:31.442+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:31.444 INFO:teuthology.orchestra.run.smithi046.stdout:3277 write 0x8225b46 thru 0x822af08 (0x53c3 bytes) 2024-09-12T07:20:31.445 INFO:teuthology.orchestra.run.smithi046.stdout:3278 write 0x3cc67fe thru 0x3ccdd2f (0x7532 bytes) 2024-09-12T07:20:31.447 INFO:teuthology.orchestra.run.smithi046.stdout:3279 write 0xac959fb thru 0xac9b808 (0x5e0e bytes) 2024-09-12T07:20:31.450 INFO:teuthology.orchestra.run.smithi046.stdout:3280 punch from 0x621fa9f to 0x6229d05, (0xa266 bytes) 2024-09-12T07:20:31.452 INFO:teuthology.orchestra.run.smithi046.stdout:3281 trunc from 0xe9d9195 to 0xcb9063b 2024-09-12T07:20:31.459 INFO:teuthology.orchestra.run.smithi046.stdout:3282 punch from 0xa241c00 to 0xa245c7c, (0x407c bytes) 2024-09-12T07:20:31.459 INFO:teuthology.orchestra.run.smithi046.stdout:3283 flatten 2024-09-12T07:20:31.464 INFO:teuthology.orchestra.run.smithi046.stdout:3284 punch from 0x83e40c7 to 0x83e4a46, (0x97f bytes) 2024-09-12T07:20:31.464 INFO:teuthology.orchestra.run.smithi046.stdout:3285 punch from 0x10480dc to 0x104c79a, (0x46be bytes) 2024-09-12T07:20:31.464 INFO:teuthology.orchestra.run.smithi046.stdout:3286 write 0x2bdaedf thru 0x2be70d7 (0xc1f9 bytes) 2024-09-12T07:20:31.465 INFO:teuthology.orchestra.run.smithi046.stdout:3287 read 0x561117f thru 0x561b41a (0xa29c bytes) 2024-09-12T07:20:31.466 INFO:teuthology.orchestra.run.smithi046.stdout:3288 read 0x509f792 thru 0x50aa9c4 (0xb233 bytes) 2024-09-12T07:20:31.466 INFO:teuthology.orchestra.run.smithi046.stdout:3290 read 0x8e6c08a thru 0x8e75872 (0x97e9 bytes) 2024-09-12T07:20:31.467 INFO:teuthology.orchestra.run.smithi046.stdout:3291 write 0xbb09db thru 0xbb1713 (0xd39 bytes) 2024-09-12T07:20:31.468 INFO:teuthology.orchestra.run.smithi046.stdout:3293 write 0x745e59f thru 0x745f244 (0xca6 bytes) 2024-09-12T07:20:31.469 INFO:teuthology.orchestra.run.smithi046.stdout:3294 read 0x2cb0821 thru 0x2cb4b0f (0x42ef bytes) 2024-09-12T07:20:31.470 INFO:teuthology.orchestra.run.smithi046.stdout:3295 writesame 0xc11310 thru 0xc16546 data_size 0x1(0x5237 bytes) 2024-09-12T07:20:31.481 INFO:teuthology.orchestra.run.smithi046.stdout:3296 write 0x2dc9b09 thru 0x2dd0efd (0x73f5 bytes) 2024-09-12T07:20:31.482 INFO:teuthology.orchestra.run.smithi046.stdout:3297 write 0xa8c9e7c thru 0xa8d45ed (0xa772 bytes) 2024-09-12T07:20:31.484 INFO:teuthology.orchestra.run.smithi046.stdout:3298 writesame 0x32679a2 thru 0x32774ab data_size 0x3(0xfb0a bytes) 2024-09-12T07:20:31.508 INFO:teuthology.orchestra.run.smithi046.stdout:3299 trunc from 0xcb9063b to 0x202e9d8 2024-09-12T07:20:31.509 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:31.510 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:31.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:31.520 INFO:teuthology.orchestra.run.smithi046.stdout:3300 write 0x17978ff thru 0x17a6144 (0xe846 bytes) 2024-09-12T07:20:31.533 INFO:teuthology.orchestra.run.smithi046.stdout:3301 trunc from 0x202e9d8 to 0xbc46443 2024-09-12T07:20:31.538 INFO:teuthology.orchestra.run.smithi046.stdout:3302 read 0x2cd883a thru 0x2ce043e (0x7c05 bytes) 2024-09-12T07:20:31.539 INFO:teuthology.orchestra.run.smithi046.stdout:3305 trunc from 0xbc46443 to 0xaff95d8 2024-09-12T07:20:31.544 INFO:teuthology.orchestra.run.smithi046.stdout:3306 read 0x10d59b8 thru 0x10d673f (0xd88 bytes) 2024-09-12T07:20:31.545 INFO:teuthology.orchestra.run.smithi046.stdout:3307 writesame 0x384c31c thru 0x3858854 data_size 0x1(0xc539 bytes) 2024-09-12T07:20:31.558 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:31.560+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:31.567 INFO:teuthology.orchestra.run.smithi046.stdout:3311 read 0x54b5a85 thru 0x54c35d3 (0xdb4f bytes) 2024-09-12T07:20:31.567 INFO:teuthology.orchestra.run.smithi046.stdout:3314 write 0x2a603c5 thru 0x2a63522 (0x315e bytes) 2024-09-12T07:20:31.570 INFO:teuthology.orchestra.run.smithi046.stdout:3316 read 0x71a4e3c thru 0x71b4d4e (0xff13 bytes) 2024-09-12T07:20:31.577 INFO:teuthology.orchestra.run.smithi046.stdout:3317 write 0xb6b97d6 thru 0xb6c6548 (0xcd73 bytes) 2024-09-12T07:20:31.580 INFO:teuthology.orchestra.run.smithi046.stdout:3318 punch from 0x4050450 to 0x405f86e, (0xf41e bytes) 2024-09-12T07:20:31.580 INFO:teuthology.orchestra.run.smithi046.stdout:3320 read 0xa63670 thru 0xa6ef21 (0xb8b2 bytes) 2024-09-12T07:20:31.581 INFO:teuthology.orchestra.run.smithi046.stdout:3321 read 0x7cb269b thru 0x7cb3ad6 (0x143c bytes) 2024-09-12T07:20:31.581 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.581 INFO:teuthology.orchestra.run.smithi046.stdout:3325 write 0x5b8b2c7 thru 0x5b92e41 (0x7b7b bytes) 2024-09-12T07:20:31.588 INFO:teuthology.orchestra.run.smithi046.stdout:3326 trunc from 0xb6c6549 to 0x780437d 2024-09-12T07:20:31.595 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.601 INFO:teuthology.orchestra.run.smithi046.stdout:3328 trunc from 0x780437d to 0x1b9f0d3 2024-09-12T07:20:31.616 INFO:teuthology.orchestra.run.smithi046.stdout:3330 read 0x17e9213 thru 0x17f0b7b (0x7969 bytes) 2024-09-12T07:20:31.617 INFO:teuthology.orchestra.run.smithi046.stdout:3331 read 0x1323dad thru 0x133381e (0xfa72 bytes) 2024-09-12T07:20:31.627 INFO:teuthology.orchestra.run.smithi046.stdout:3332 write 0x5512e7c thru 0x551749b (0x4620 bytes) 2024-09-12T07:20:31.632 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:31.633+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:31.633 INFO:teuthology.orchestra.run.smithi046.stdout:3333 trunc from 0x551749c to 0x9431a4c 2024-09-12T07:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T07:20:31.639+0000 7fa0af7fe640 0 --2- 172.21.15.46:0/245166158 >> v2:172.21.15.46:3300/0 conn(0x7fa0b00b1d30 0x7fa0b0144da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-12T07:20:31.638 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.639 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.639 INFO:teuthology.orchestra.run.smithi046.stdout:3336 read 0x351bba5 thru 0x35298f4 (0xdd50 bytes) 2024-09-12T07:20:31.639 INFO:teuthology.orchestra.run.smithi046.stdout:3337 trunc from 0x9431a4c to 0x992bbd5 2024-09-12T07:20:31.645 INFO:teuthology.orchestra.run.smithi046.stdout:3338 punch from 0x13cc734 to 0x13d7bad, (0xb479 bytes) 2024-09-12T07:20:31.650 INFO:teuthology.orchestra.run.smithi046.stdout:3341 trunc from 0x992bbd5 to 0x537bde6 2024-09-12T07:20:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:3342 writesame 0x48eb86e thru 0x48f5f25 data_size 0x2c(0xa6b8 bytes) 2024-09-12T07:20:31.658 INFO:teuthology.orchestra.run.smithi046.stdout:3343 read 0x4dab42a thru 0x4db2ce7 (0x78be bytes) 2024-09-12T07:20:31.659 INFO:teuthology.orchestra.run.smithi046.stdout:3345 read 0x60a9df thru 0x60c7ca (0x1dec bytes) 2024-09-12T07:20:31.659 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.659 INFO:teuthology.orchestra.run.smithi046.stdout:3350 read 0x1586374 thru 0x158db97 (0x7824 bytes) 2024-09-12T07:20:31.659 INFO:teuthology.orchestra.run.smithi046.stdout:3351 punch from 0x312b24a to 0x3135dee, (0xaba4 bytes) 2024-09-12T07:20:31.660 INFO:teuthology.orchestra.run.smithi046.stdout:3352 read 0x21f69cb thru 0x21fa6d0 (0x3d06 bytes) 2024-09-12T07:20:31.661 INFO:teuthology.orchestra.run.smithi046.stdout:3353 trunc from 0x537bde6 to 0x3d9ec71 2024-09-12T07:20:31.668 INFO:teuthology.orchestra.run.smithi046.stdout:3355 read 0xcb249c thru 0xcb9913 (0x7478 bytes) 2024-09-12T07:20:31.674 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:31.674 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:31.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:31.674 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:31.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.674 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:20:31.674 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:31.674 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 34.061801392, 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:31.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:20:31.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:20:31.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:20:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:20:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:20:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:20:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:20:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:20:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.926341136000001, 2024-09-12T07:20:31.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:31.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:20:31.683 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.922114014999998, 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:31.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:31.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:20:31.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.688 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.194363360000001, 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:31.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:20:31.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.105386770999999, 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.693 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:20:31.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68853 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'43, mlcod=43'39)", 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.094394183999999, 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017793166999999999, 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580719+0000", 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4720000000000001e-06 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580732+0000", 2024-09-12T07:20:31.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2926e-05 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580746+0000", 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3291e-05 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580755+0000", 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6709999999999994e-06 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583766+0000", 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003010432 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583794+0000", 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.811e-05 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598408+0000", 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014614472 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598511+0000", 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102793 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.073835441, 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.702 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:20:31.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.070634904999999, 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:20:31.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:20:31.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:20:31.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.067389157000001, 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:20:31.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.065147326000002, 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:20:31.715 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:20:31.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:20:31.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69559 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'29, mlcod=41'27)", 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 27.990816278, 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018119258999999999, 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:31.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684297+0000", 2024-09-12T07:20:31.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.192e-06 2024-09-12T07:20:31.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691509+0000", 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072122239999999997 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691524+0000", 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4562999999999999e-05 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691533+0000", 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1069999999999994e-06 2024-09-12T07:20:31.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691598+0000", 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4666999999999996e-05 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691620+0000", 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2092000000000001e-05 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702352+0000", 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010731785000000001 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702415+0000", 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3628999999999997e-05 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69560 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'31, mlcod=41'27)", 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:31.724 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 27.983250155, 2024-09-12T07:20:31.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016047555000000002, 2024-09-12T07:20:31.725 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.725 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.725 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.725 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:31.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:31.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691863+0000", 2024-09-12T07:20:31.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0299999999999999e-07 2024-09-12T07:20:31.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691897+0000", 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4765999999999999e-05 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691926+0000", 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8413000000000001e-05 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691934+0000", 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3839999999999997e-06 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694739+0000", 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028043640000000002 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694773+0000", 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4514000000000002e-05 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707854+0000", 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013080591000000001 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707910+0000", 2024-09-12T07:20:31.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.592e-05 2024-09-12T07:20:31.729 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.729 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.729 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.729 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "pg_update_log_missing(3.1a epoch 43/24 rep_tid 362 entries 43'32 (0'0) error 3:5ec4a676:::rbd_data.1160f022dd61.0000000000000019:head by client.4448.0:69639 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 41'27)", 2024-09-12T07:20:31.729 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:31.729 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 27.982722441, 2024-09-12T07:20:31.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015537618, 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692497+0000", 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107154 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692519+0000", 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1043000000000002e-05 2024-09-12T07:20:31.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692542+0000", 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3866000000000002e-05 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702496+0000", 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0099539169999999996 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707927+0000", 2024-09-12T07:20:31.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005431038 2024-09-12T07:20:31.733 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.733 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.733 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.733 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:20:31.733 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:31.733 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 27.974040122000002, 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:31.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:20:31.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:20:31.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:20:31.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:20:31.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 27.970212033999999, 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:20:31.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:20:31.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:20:31.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:20:31.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.742 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89144 3.a e53/24 3:54996542:::rbd_data.11607a56846.0000000000000001:head v 53'83, mlcod=53'81)", 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 17.982055280000001, 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018298937000000001, 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:31.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693059+0000", 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.565e-06 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696238+0000", 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0031783269999999999 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696265+0000", 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6982e-05 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696279+0000", 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4438e-05 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696366+0000", 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6978000000000006e-05 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696397+0000", 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0406999999999999e-05 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711297+0000", 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014900689999999999 2024-09-12T07:20:31.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711356+0000", 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.855e-05 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89145 3.13 e53/24 3:ca7c3842:::rbd_data.11607a56846.0000000000000002:head v 53'202, mlcod=53'199)", 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 17.981023937, 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019835603, 2024-09-12T07:20:31.747 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694091+0000", 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4990000000000001e-06 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697616+0000", 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035256749999999998 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697635+0000", 2024-09-12T07:20:31.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8593999999999998e-05 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697646+0000", 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1436e-05 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697717+0000", 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0679000000000005e-05 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697748+0000", 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0723999999999999e-05 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713886+0000", 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016138550000000002 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713924+0000", 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7446e-05 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.751 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.752 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:20:31.752 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:31.752 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.9882992530000001, 2024-09-12T07:20:31.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:20:31.752 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.752 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.752 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.752 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:20:31.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:20:31.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:31.756 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.976485609, 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:31.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:20:31.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:20:31.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:20:31.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:20:31.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:20:31.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:20:31.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.9751495620000004, 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:31.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:20:31.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:20:31.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:31.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:20:31.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:20:31.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:31.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:31.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:31.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:20:31.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:20:31.765 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.765 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.765 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.765 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:31.765 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:31.765 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:31.767 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:3356 write 0xa9e6284 thru 0xa9f2cb3 (0xca30 bytes) 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:3359 read 0x7035cd6 thru 0x704537f (0xf6aa bytes) 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:3360 punch from 0x266dd86 to 0x267495d, (0x6bd7 bytes) 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:3362 trunc from 0xa9f2cb4 to 0x9bb7589 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:3364 trunc from 0x9bb7589 to 0x14f22f2 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:3368 writesame 0x78ddffa thru 0x78e5116 data_size 0x1(0x711d bytes) 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:3370 trunc from 0x78e5117 to 0x7a7aca9 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:3371 write 0x904159d thru 0x9049e9d (0x8901 bytes) 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:3373 trunc from 0x9049e9e to 0xbf890ca 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:3374 write 0xc4de07a thru 0xc4e0e9d (0x2e24 bytes) 2024-09-12T07:20:31.769 INFO:teuthology.orchestra.run.smithi046.stdout:3375 write 0x86493f9 thru 0x8652bfd (0x9805 bytes) 2024-09-12T07:20:31.770 INFO:teuthology.orchestra.run.smithi046.stdout:3378 read 0x5bb3bc4 thru 0x5bc2375 (0xe7b2 bytes) 2024-09-12T07:20:31.770 INFO:teuthology.orchestra.run.smithi046.stdout:3379 clone 25 order 22 su 2097152 sc 2 2024-09-12T07:20:31.770 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:31.768+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:31.867 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:31.869+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:31.915 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:31.915 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:31.915 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:31.915 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:31.923 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:31.968 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:31.970+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:32.069 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:32.070+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:32.070 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:32.071 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:32.079 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:32.170 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:32.171+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:32.227 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:32.227 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:32.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:32.227 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:32.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.227 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.227 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:32.227 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.746308595000002, 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:20:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.746281715000002, 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.232 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.236 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.236 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:20:32.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:20:32.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.236 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.236 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:20:32.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:20:32.236 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.236 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.236 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.237 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.237 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:32.237 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.746257694999997, 2024-09-12T07:20:32.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:20:32.238 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.238 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.238 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.238 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.238 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.238 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.238 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:20:32.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.238 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:20:32.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:20:32.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:20:32.242 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.242 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.242 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.242 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.242 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:32.242 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.746230048999998, 2024-09-12T07:20:32.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:20:32.242 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.242 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:20:32.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:20:32.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:20:32.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:20:32.246 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.246 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.746206587000003, 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:20:32.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:20:32.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.746184059999997, 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:20:32.251 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:20:32.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:20:32.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:20:32.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:20:32.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:20:32.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.746162257000002, 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.256 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.260 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.261 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:32.261 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.739528477, 2024-09-12T07:20:32.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:20:32.261 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.261 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.261 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.261 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.261 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.261 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.261 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:20:32.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:32.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:20:32.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:20:32.264 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.739508469, 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.265 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.266 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:20:32.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.266 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:32.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:20:32.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.733025437000002, 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.270 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:20:32.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.271 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:32.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:32.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:20:32.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.732990149999999, 2024-09-12T07:20:32.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:32.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:20:32.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.732967516999999, 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.279 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:32.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:20:32.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.283 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.732913461999999, 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:32.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.732894520999999, 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:20:32.288 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.289 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.289 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.289 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.289 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.289 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.289 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:20:32.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.289 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:32.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:20:32.292 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.732882584000002, 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.293 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:20:32.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.297 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.732870968999997, 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:20:32.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.725858361, 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.302 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:20:32.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.306 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.725834646000003, 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.307 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:20:32.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:20:32.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.725703789000001, 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:20:32.311 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:20:32.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:32.315 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.762032998999999, 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:20:32.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:32.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:20:32.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:20:32.321 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.321 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.321 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.321 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.321 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.321 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.6 to 1 (10000) 2024-09-12T07:20:32.323 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:32.324 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:32.324+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:32.324 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is set 2024-09-12T07:20:32.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-12T07:20:32.423 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:32.425+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:32.472 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:32.472 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:32.481 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:32.484 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-12T07:20:32.525 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:32.526+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:32.591 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:32.591 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:32.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:32.643 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:32.645+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:32.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:20:32.708 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:20:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.179147321000002, 2024-09-12T07:20:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:20:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:20:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:20:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.327349654999999, 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:20:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.327226377999999, 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.327022986999999, 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:20:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.326887159999998, 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:20:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:20:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.326560504, 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:20:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:20:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:20:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:20:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:20:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:20:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:20:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:20:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:20:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:20:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:20:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:20:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.326521616000001, 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:20:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:20:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.326154742, 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:20:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.326019885999997, 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:20:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.325649994999999, 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:20:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.325459797000001, 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:20:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:20:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.325239717999999, 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:20:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:20:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.325096053000003, 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.324869880999998, 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:20:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:20:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:20:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:20:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:20:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:20:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:20:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:20:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.324669522999997, 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:20:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.324261640000003, 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.323795758000003, 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.322193548000001, 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:20:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:20:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.321659513, 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.065461442999997, 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:20:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:20:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:20:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:32.800 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:32.801 INFO:teuthology.orchestra.run.smithi046.stdout:3380 read 0xa2f8738 thru 0xa2fd185 (0x4a4e bytes) 2024-09-12T07:20:32.801 INFO:teuthology.orchestra.run.smithi046.stdout:3381 trunc from 0xc4e0e9e to 0x6386f88 2024-09-12T07:20:32.801 INFO:teuthology.orchestra.run.smithi046.stdout:3387 writesame 0xc8969 thru 0xd7bc1 data_size 0x1(0xf259 bytes) 2024-09-12T07:20:32.802 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:32.802+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:32.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:32.803 INFO:teuthology.orchestra.run.smithi046.stdout:3388 writesame 0xb3b694e thru 0xb3b90ad data_size 0xa(0x2760 bytes) 2024-09-12T07:20:32.803 INFO:teuthology.orchestra.run.smithi046.stdout:3389 writesame 0x3e2001f thru 0x3e2c3f9 data_size 0x1b(0xc3db bytes) 2024-09-12T07:20:32.805 INFO:teuthology.orchestra.run.smithi046.stdout:3390 writesame 0x8ec6c9b thru 0x8ec9354 data_size 0x1(0x26ba bytes) 2024-09-12T07:20:32.811 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:32.811 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:32.812 INFO:teuthology.orchestra.run.smithi046.stdout:3391 write 0x5a8e13b thru 0x5a91113 (0x2fd9 bytes) 2024-09-12T07:20:32.815 INFO:teuthology.orchestra.run.smithi046.stdout:3393 write 0x1c5aa11 thru 0x1c5fdda (0x53ca bytes) 2024-09-12T07:20:32.818 INFO:teuthology.orchestra.run.smithi046.stdout:3394 read 0x1a02cba thru 0x1a09cde (0x7025 bytes) 2024-09-12T07:20:32.818 INFO:teuthology.orchestra.run.smithi046.stdout:3396 read 0x1b02ccb thru 0x1b10019 (0xd34f bytes) 2024-09-12T07:20:32.818 INFO:teuthology.orchestra.run.smithi046.stdout:3397 punch from 0xcfc710 to 0xcfd846, (0x1136 bytes) 2024-09-12T07:20:32.818 INFO:teuthology.orchestra.run.smithi046.stdout:3398 punch from 0xf58f44 to 0xf60797, (0x7853 bytes) 2024-09-12T07:20:32.818 INFO:teuthology.orchestra.run.smithi046.stdout:3402 punch from 0x6b2849c to 0x6b2f54b, (0x70af bytes) 2024-09-12T07:20:32.818 INFO:teuthology.orchestra.run.smithi046.stdout:3403 write 0x17fa00a thru 0x17fcd81 (0x2d78 bytes) 2024-09-12T07:20:32.821 INFO:teuthology.orchestra.run.smithi046.stdout:3404 punch from 0x73d747c to 0x73e4ce1, (0xd865 bytes) 2024-09-12T07:20:32.821 INFO:teuthology.orchestra.run.smithi046.stdout:3405 read 0xad6b08f thru 0xad76c13 (0xbb85 bytes) 2024-09-12T07:20:32.821 INFO:teuthology.orchestra.run.smithi046.stdout:3406 write 0xa1dc603 thru 0xa1ddc2f (0x162d bytes) 2024-09-12T07:20:32.823 INFO:teuthology.orchestra.run.smithi046.stdout:3407 writesame 0x9645cf7 thru 0x965304f data_size 0x5(0xd359 bytes) 2024-09-12T07:20:32.832 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:32.834 INFO:teuthology.orchestra.run.smithi046.stdout:3409 trunc from 0xb3b90ae to 0x8e44228 2024-09-12T07:20:32.845 INFO:teuthology.orchestra.run.smithi046.stdout:3410 writesame 0x8da90ce thru 0x8dacc8a data_size 0x1(0x3bbd bytes) 2024-09-12T07:20:32.854 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:32.855 INFO:teuthology.orchestra.run.smithi046.stdout:3411 read 0x824d662 thru 0x8257c5b (0xa5fa bytes) 2024-09-12T07:20:32.855 INFO:teuthology.orchestra.run.smithi046.stdout:3412 write 0x2602292 thru 0x2605f9c (0x3d0b bytes) 2024-09-12T07:20:32.859 INFO:teuthology.orchestra.run.smithi046.stdout:3414 write 0x5fef026 thru 0x5ff3a42 (0x4a1d bytes) 2024-09-12T07:20:32.860 INFO:teuthology.orchestra.run.smithi046.stdout:3415 write 0x3f6944a thru 0x3f707ce (0x7385 bytes) 2024-09-12T07:20:32.862 INFO:teuthology.orchestra.run.smithi046.stdout:3416 write 0x2cf979e thru 0x2d00020 (0x6883 bytes) 2024-09-12T07:20:32.868 INFO:teuthology.orchestra.run.smithi046.stdout:3418 trunc from 0x8e44228 to 0x5f3e3ce 2024-09-12T07:20:32.872 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:32.889 INFO:teuthology.orchestra.run.smithi046.stdout:3419 trunc from 0x5f3e3ce to 0xeaa0c51 2024-09-12T07:20:32.889 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:32.890 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:32.894 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:32.894 INFO:teuthology.orchestra.run.smithi046.stdout:3421 read 0xea3e6ab thru 0xea4c61e (0xdf74 bytes) 2024-09-12T07:20:32.894 INFO:teuthology.orchestra.run.smithi046.stdout:3422 write 0x222804c thru 0x222d0ac (0x5061 bytes) 2024-09-12T07:20:32.895 INFO:teuthology.orchestra.run.smithi046.stdout:3423 read 0x503e1ea thru 0x50432eb (0x5102 bytes) 2024-09-12T07:20:32.895 INFO:teuthology.orchestra.run.smithi046.stdout:3425 read 0xde8eb5d thru 0xde9cb5a (0xdffe bytes) 2024-09-12T07:20:32.896 INFO:teuthology.orchestra.run.smithi046.stdout:3426 clone 26 order 22 su 32768 sc 8 2024-09-12T07:20:32.902 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:32.903+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:32.907 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:32.907 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:32.925 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:32.925 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:32.942 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:32.954 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:32.954 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:32.962 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:33.003 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:33.004+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:33.104 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:33.106+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:33.109 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:33.109 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:33.117 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:33.205 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:33.206+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:33.265 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:33.265 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:33.273 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:33.304 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.7 to 1 (10000) 2024-09-12T07:20:33.306 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:33.307+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:33.321 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-12T07:20:33.366 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone25 from 0xeaa0c51 (overlap 0x5f3e3ce) to 0x986394 2024-09-12T07:20:33.400 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:33.400 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:33.406 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:33.408+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:33.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:33.432 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-12T07:20:33.508 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:33.509+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:33.562 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:33.562 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:33.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:33.613 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:33.614+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:33.653 INFO:teuthology.orchestra.run.smithi046.stdout:3427 write 0x4ea1181 thru 0x4eac2ec (0xb16c bytes) 2024-09-12T07:20:33.677 INFO:teuthology.orchestra.run.smithi046.stdout:3428 writesame 0x54405ac thru 0x544a718 data_size 0x5f(0xa16d bytes) 2024-09-12T07:20:33.709 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:33.710+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:33.714 INFO:teuthology.orchestra.run.smithi046.stdout:3429 read 0x6af9285 thru 0x6b00cfd (0x7a79 bytes) 2024-09-12T07:20:33.714 INFO:teuthology.orchestra.run.smithi046.stdout:3430 punch from 0x4b7c17b to 0x4b80bfe, (0x4a83 bytes) 2024-09-12T07:20:33.714 INFO:teuthology.orchestra.run.smithi046.stdout:3431 write 0xd647a40 thru 0xd656c66 (0xf227 bytes) 2024-09-12T07:20:33.720 INFO:teuthology.orchestra.run.smithi046.stdout:3432 write 0x52c17de thru 0x52cdae4 (0xc307 bytes) 2024-09-12T07:20:33.722 INFO:teuthology.orchestra.run.smithi046.stdout:3433 read 0xbd619d5 thru 0xbd6563a (0x3c66 bytes) 2024-09-12T07:20:33.722 INFO:teuthology.orchestra.run.smithi046.stdout:3434 read 0x973e8a5 thru 0x9747822 (0x8f7e bytes) 2024-09-12T07:20:33.722 INFO:teuthology.orchestra.run.smithi046.stdout:3435 write 0x9c3eb77 thru 0x9c497e0 (0xac6a bytes) 2024-09-12T07:20:33.728 INFO:teuthology.orchestra.run.smithi046.stdout:3436 write 0x735526a thru 0x735b6cc (0x6463 bytes) 2024-09-12T07:20:33.732 INFO:teuthology.orchestra.run.smithi046.stdout:3437 write 0x54522bf thru 0x5460da8 (0xeaea bytes) 2024-09-12T07:20:33.743 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:33.743 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:33.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:33.757 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:33.757 INFO:teuthology.orchestra.run.smithi046.stdout:3440 punch from 0x3b75bcd to 0x3b76e31, (0x1264 bytes) 2024-09-12T07:20:33.757 INFO:teuthology.orchestra.run.smithi046.stdout:3441 read 0x1859fb2 thru 0x1863c23 (0x9c72 bytes) 2024-09-12T07:20:33.757 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:33.757 INFO:teuthology.orchestra.run.smithi046.stdout:3444 write 0x60fc7a5 thru 0x6106f4d (0xa7a9 bytes) 2024-09-12T07:20:33.761 INFO:teuthology.orchestra.run.smithi046.stdout:3446 writesame 0x58d7df8 thru 0x58d868b data_size 0x12(0x894 bytes) 2024-09-12T07:20:33.763 INFO:teuthology.orchestra.run.smithi046.stdout:3447 write 0xbeb8765 thru 0xbec1597 (0x8e33 bytes) 2024-09-12T07:20:33.767 INFO:teuthology.orchestra.run.smithi046.stdout:3448 read 0xce20e86 thru 0xce287f0 (0x796b bytes) 2024-09-12T07:20:33.767 INFO:teuthology.orchestra.run.smithi046.stdout:3449 write 0x93326a thru 0x93967e (0x6415 bytes) 2024-09-12T07:20:33.790 INFO:teuthology.orchestra.run.smithi046.stdout:3450 write 0x264ac2e thru 0x265889e (0xdc71 bytes) 2024-09-12T07:20:33.806 INFO:teuthology.orchestra.run.smithi046.stdout:3452 punch from 0x1293a58 to 0x12a00b9, (0xc661 bytes) 2024-09-12T07:20:33.810 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:33.812+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:33.816 INFO:teuthology.orchestra.run.smithi046.stdout:3453 trunc from 0xeaa0c51 to 0x5675d6b 2024-09-12T07:20:33.853 INFO:teuthology.orchestra.run.smithi046.stdout:3454 trunc from 0x5675d6b to 0x299ae16 2024-09-12T07:20:33.894 INFO:teuthology.orchestra.run.smithi046.stdout:3455 write 0x5b2d770 thru 0x5b34c32 (0x74c3 bytes) 2024-09-12T07:20:33.899 INFO:teuthology.orchestra.run.smithi046.stdout:3457 trunc from 0x5b34c33 to 0x4b978ca 2024-09-12T07:20:33.900 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:33.900 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:33.906 INFO:teuthology.orchestra.run.smithi046.stdout:3458 punch from 0x15d3221 to 0x15d6716, (0x34f5 bytes) 2024-09-12T07:20:33.906 INFO:teuthology.orchestra.run.smithi046.stdout:3459 read 0x4480fea thru 0x448d46f (0xc486 bytes) 2024-09-12T07:20:33.910 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:33.911 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:33.913+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:33.913 INFO:teuthology.orchestra.run.smithi046.stdout:3460 writesame 0x56ac68a thru 0x56b8480 data_size 0x1(0xbdf7 bytes) 2024-09-12T07:20:33.915 INFO:teuthology.orchestra.run.smithi046.stdout:3461 write 0x684778 thru 0x68b7b8 (0x7041 bytes) 2024-09-12T07:20:33.942 INFO:teuthology.orchestra.run.smithi046.stdout:3462 read 0x15585ec thru 0x156520a (0xcc1f bytes) 2024-09-12T07:20:33.948 INFO:teuthology.orchestra.run.smithi046.stdout:3463 trunc from 0x56b8481 to 0x47717d 2024-09-12T07:20:33.985 INFO:teuthology.orchestra.run.smithi046.stdout:3465 punch from 0xa4942 to 0xb3222, (0xe8e0 bytes) 2024-09-12T07:20:34.007 INFO:teuthology.orchestra.run.smithi046.stdout:3467 write 0xe5c2524 thru 0xe5c507f (0x2b5c bytes) 2024-09-12T07:20:34.010 INFO:teuthology.orchestra.run.smithi046.stdout:3468 writesame 0x2240588 thru 0x2250468 data_size 0x1(0xfee1 bytes) 2024-09-12T07:20:34.013 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:34.014+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:34.039 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:34.039 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:34.039 INFO:teuthology.orchestra.run.smithi046.stdout:3469 write 0x8eecfd2 thru 0x8ef14f9 (0x4528 bytes) 2024-09-12T07:20:34.043 INFO:teuthology.orchestra.run.smithi046.stdout:3470 writesame 0x77a9104 thru 0x77b1cd0 data_size 0x1(0x8bcd bytes) 2024-09-12T07:20:34.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:34.061 INFO:teuthology.orchestra.run.smithi046.stdout:3471 write 0xb96552a thru 0xb96b843 (0x631a bytes) 2024-09-12T07:20:34.064 INFO:teuthology.orchestra.run.smithi046.stdout:3472 read 0x9274411 thru 0x927e740 (0xa330 bytes) 2024-09-12T07:20:34.065 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.070 INFO:teuthology.orchestra.run.smithi046.stdout:3476 trunc from 0xe5c5080 to 0x9d854bd 2024-09-12T07:20:34.090 INFO:teuthology.orchestra.run.smithi046.stdout:3478 writesame 0xca9b1ba thru 0xcaa57ec data_size 0x1(0xa633 bytes) 2024-09-12T07:20:34.090 INFO:teuthology.orchestra.run.smithi046.stdout:3479 read 0x582166f thru 0x5830d14 (0xf6a6 bytes) 2024-09-12T07:20:34.090 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.090 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.090 INFO:teuthology.orchestra.run.smithi046.stdout:3482 writesame 0xaaa8aee thru 0xaaaeab8 data_size 0x1(0x5fcb bytes) 2024-09-12T07:20:34.103 INFO:teuthology.orchestra.run.smithi046.stdout:3484 writesame 0x7f5dd7e thru 0x7f6223b data_size 0x15(0x44be bytes) 2024-09-12T07:20:34.107 INFO:teuthology.orchestra.run.smithi046.stdout:3486 write 0xa5fd1fe thru 0xa609d8a (0xcb8d bytes) 2024-09-12T07:20:34.112 INFO:teuthology.orchestra.run.smithi046.stdout:3487 punch from 0x2943779 to 0x294b63d, (0x7ec4 bytes) 2024-09-12T07:20:34.112 INFO:teuthology.orchestra.run.smithi046.stdout:3488 write 0xef39c5 thru 0xf0142b (0xda67 bytes) 2024-09-12T07:20:34.114 INFO:teuthology.orchestra.run.smithi046.stdout:3489 write 0x1968a61 thru 0x1974efb (0xc49b bytes) 2024-09-12T07:20:34.114 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:34.115+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:34.115 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.115 INFO:teuthology.orchestra.run.smithi046.stdout:3491 read 0x8adfef7 thru 0x8ae5406 (0x5510 bytes) 2024-09-12T07:20:34.115 INFO:teuthology.orchestra.run.smithi046.stdout:3492 write 0x9b2a6f8 thru 0x9b2f66b (0x4f74 bytes) 2024-09-12T07:20:34.122 INFO:teuthology.orchestra.run.smithi046.stdout:3493 trunc from 0xcaa57ed to 0x7fa49ae 2024-09-12T07:20:34.133 INFO:teuthology.orchestra.run.smithi046.stdout:3495 read 0x705e862 thru 0x706417f (0x591e bytes) 2024-09-12T07:20:34.134 INFO:teuthology.orchestra.run.smithi046.stdout:3496 punch from 0x7f2c4cb to 0x7f3a631, (0xe166 bytes) 2024-09-12T07:20:34.134 INFO:teuthology.orchestra.run.smithi046.stdout:3497 read 0x3544ad3 thru 0x35453dd (0x90b bytes) 2024-09-12T07:20:34.134 INFO:teuthology.orchestra.run.smithi046.stdout:3498 read 0x13b23e8 thru 0x13c1e07 (0xfa20 bytes) 2024-09-12T07:20:34.134 INFO:teuthology.orchestra.run.smithi046.stdout:3499 write 0xcc5d75 thru 0xccbbe2 (0x5e6e bytes) 2024-09-12T07:20:34.136 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.136 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.136 INFO:teuthology.orchestra.run.smithi046.stdout:3502 punch from 0x20e101d to 0x20e20ee, (0x10d1 bytes) 2024-09-12T07:20:34.136 INFO:teuthology.orchestra.run.smithi046.stdout:3503 write 0x5cb9b4d thru 0x5cc877a (0xec2e bytes) 2024-09-12T07:20:34.141 INFO:teuthology.orchestra.run.smithi046.stdout:3504 punch from 0x5ae7a57 to 0x5af2034, (0xa5dd bytes) 2024-09-12T07:20:34.141 INFO:teuthology.orchestra.run.smithi046.stdout:3505 read 0x24a3412 thru 0x24ac433 (0x9022 bytes) 2024-09-12T07:20:34.141 INFO:teuthology.orchestra.run.smithi046.stdout:3506 read 0x24de886 thru 0x24e9a9b (0xb216 bytes) 2024-09-12T07:20:34.141 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.141 INFO:teuthology.orchestra.run.smithi046.stdout:3508 writesame 0x7594151 thru 0x759b32e data_size 0x19(0x71de bytes) 2024-09-12T07:20:34.144 INFO:teuthology.orchestra.run.smithi046.stdout:3509 read 0xd67968 thru 0xd69fb8 (0x2651 bytes) 2024-09-12T07:20:34.151 INFO:teuthology.orchestra.run.smithi046.stdout:3510 writesame 0x879ce37 thru 0x87a3e2c data_size 0x33(0x6ff6 bytes) 2024-09-12T07:20:34.151 INFO:teuthology.orchestra.run.smithi046.stdout:3511 punch from 0x766877b to 0x766b60c, (0x2e91 bytes) 2024-09-12T07:20:34.151 INFO:teuthology.orchestra.run.smithi046.stdout:3512 read 0x864608a thru 0x86513f4 (0xb36b bytes) 2024-09-12T07:20:34.151 INFO:teuthology.orchestra.run.smithi046.stdout:3513 flatten 2024-09-12T07:20:34.156 INFO:teuthology.orchestra.run.smithi046.stdout:3514 read 0x2710879 thru 0x2715a94 (0x521c bytes) 2024-09-12T07:20:34.156 INFO:teuthology.orchestra.run.smithi046.stdout:3515 writesame 0x2b3ddc3 thru 0x2b42480 data_size 0x5(0x46be bytes) 2024-09-12T07:20:34.171 INFO:teuthology.orchestra.run.smithi046.stdout:3516 write 0xcfbd365 thru 0xcfca522 (0xd1be bytes) 2024-09-12T07:20:34.176 INFO:teuthology.orchestra.run.smithi046.stdout:3517 writesame 0x8e8d806 thru 0x8e9867d data_size 0xc(0xae78 bytes) 2024-09-12T07:20:34.181 INFO:teuthology.orchestra.run.smithi046.stdout:3518 write 0x51ec2d0 thru 0x51ef4e1 (0x3212 bytes) 2024-09-12T07:20:34.184 INFO:teuthology.orchestra.run.smithi046.stdout:3519 write 0x1826a6c thru 0x1831e97 (0xb42c bytes) 2024-09-12T07:20:34.185 INFO:teuthology.orchestra.run.smithi046.stdout:3520 write 0xa03422c thru 0xa0347d0 (0x5a5 bytes) 2024-09-12T07:20:34.188 INFO:teuthology.orchestra.run.smithi046.stdout:3521 writesame 0x12a2e73 thru 0x12af653 data_size 0x1(0xc7e1 bytes) 2024-09-12T07:20:34.189 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:34.189 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:34.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:34.211 INFO:teuthology.orchestra.run.smithi046.stdout:3522 trunc from 0xcfca523 to 0xc027697 2024-09-12T07:20:34.221 INFO:teuthology.orchestra.run.smithi046.stdout:3524 write 0x23d78e4 thru 0x23e217c (0xa899 bytes) 2024-09-12T07:20:34.221 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:34.222+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:3525 read 0x95d67f2 thru 0x95e30c8 (0xc8d7 bytes) 2024-09-12T07:20:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:3528 writesame 0x3ef62e5 thru 0x3ef6450 data_size 0x2(0x16c bytes) 2024-09-12T07:20:34.228 INFO:teuthology.orchestra.run.smithi046.stdout:3529 write 0x600922b thru 0x60179a2 (0xe778 bytes) 2024-09-12T07:20:34.235 INFO:teuthology.orchestra.run.smithi046.stdout:3530 trunc from 0xc027697 to 0x85a3e0a 2024-09-12T07:20:34.248 INFO:teuthology.orchestra.run.smithi046.stdout:3531 read 0x64b61ba thru 0x64b7828 (0x166f bytes) 2024-09-12T07:20:34.248 INFO:teuthology.orchestra.run.smithi046.stdout:3532 writesame 0x5a238eb thru 0x5a2f2a1 data_size 0x1(0xb9b7 bytes) 2024-09-12T07:20:34.264 INFO:teuthology.orchestra.run.smithi046.stdout:3533 punch from 0x2877936 to 0x28859f0, (0xe0ba bytes) 2024-09-12T07:20:34.265 INFO:teuthology.orchestra.run.smithi046.stdout:3534 read 0x72e60f6 thru 0x72ef012 (0x8f1d bytes) 2024-09-12T07:20:34.275 INFO:teuthology.orchestra.run.smithi046.stdout:3536 writesame 0xb9516ab thru 0xb95f3e6 data_size 0x2(0xdd3c bytes) 2024-09-12T07:20:34.275 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.275 INFO:teuthology.orchestra.run.smithi046.stdout:3542 punch from 0xa309907 to 0xa31788e, (0xdf87 bytes) 2024-09-12T07:20:34.275 INFO:teuthology.orchestra.run.smithi046.stdout:3543 write 0x36972d6 thru 0x3697920 (0x64b bytes) 2024-09-12T07:20:34.285 INFO:teuthology.orchestra.run.smithi046.stdout:3544 write 0xd967b55 thru 0xd973719 (0xbbc5 bytes) 2024-09-12T07:20:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:3545 write 0x92de21e thru 0x92e18e4 (0x36c7 bytes) 2024-09-12T07:20:34.292 INFO:teuthology.orchestra.run.smithi046.stdout:3547 write 0xd196f49 thru 0xd1a57f8 (0xe8b0 bytes) 2024-09-12T07:20:34.296 INFO:teuthology.orchestra.run.smithi046.stdout:3548 read 0x1486b03 thru 0x148ebb7 (0x80b5 bytes) 2024-09-12T07:20:34.297 INFO:teuthology.orchestra.run.smithi046.stdout:3553 read 0x70ce425 thru 0x70d9bc0 (0xb79c bytes) 2024-09-12T07:20:34.297 INFO:teuthology.orchestra.run.smithi046.stdout:3554 read 0x49c6784 thru 0x49d416b (0xd9e8 bytes) 2024-09-12T07:20:34.297 INFO:teuthology.orchestra.run.smithi046.stdout:3555 writesame 0x303f453 thru 0x3043b54 data_size 0x3d(0x4702 bytes) 2024-09-12T07:20:34.300 INFO:teuthology.orchestra.run.smithi046.stdout:3556 punch from 0x56a7799 to 0x56aa9bc, (0x3223 bytes) 2024-09-12T07:20:34.300 INFO:teuthology.orchestra.run.smithi046.stdout:3557 read 0x60362df thru 0x603a2df (0x4001 bytes) 2024-09-12T07:20:34.300 INFO:teuthology.orchestra.run.smithi046.stdout:3558 read 0x891ed92 thru 0x892de81 (0xf0f0 bytes) 2024-09-12T07:20:34.300 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.300 INFO:teuthology.orchestra.run.smithi046.stdout:3560 write 0x6793bd5 thru 0x679aa3a (0x6e66 bytes) 2024-09-12T07:20:34.302 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.302 INFO:teuthology.orchestra.run.smithi046.stdout:3563 write 0x75d94fc thru 0x75e4bc2 (0xb6c7 bytes) 2024-09-12T07:20:34.303 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.303 INFO:teuthology.orchestra.run.smithi046.stdout:3566 punch from 0x4fb70bd to 0x4fb71e7, (0x12a bytes) 2024-09-12T07:20:34.304 INFO:teuthology.orchestra.run.smithi046.stdout:3567 read 0x5f26d7e thru 0x5f33dbe (0xd041 bytes) 2024-09-12T07:20:34.304 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.305 INFO:teuthology.orchestra.run.smithi046.stdout:3569 trunc from 0xd97371a to 0xe55b97a 2024-09-12T07:20:34.308 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.3 to 1 (10000) 2024-09-12T07:20:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:3570 read 0xb6aff34 thru 0xb6bf5db (0xf6a8 bytes) 2024-09-12T07:20:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:3571 write 0x5af86b1 thru 0x5b07003 (0xe953 bytes) 2024-09-12T07:20:34.315 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:34.317+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:34.318 INFO:teuthology.orchestra.run.smithi046.stdout:3573 write 0xd7f5313 thru 0xd7fe214 (0x8f02 bytes) 2024-09-12T07:20:34.322 INFO:teuthology.orchestra.run.smithi046.stdout:3574 writesame 0x3f65f3c thru 0x3f69f2d data_size 0x5(0x3ff2 bytes) 2024-09-12T07:20:34.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:20:34.327 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-12T07:20:34.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:20:34.327 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.327 INFO:teuthology.orchestra.run.smithi046.stdout:3576 punch from 0x36f19d0 to 0x36fe3a0, (0xc9d0 bytes) 2024-09-12T07:20:34.327 INFO:teuthology.orchestra.run.smithi046.stdout:3577 read 0xcbe088f thru 0xcbee6a3 (0xde15 bytes) 2024-09-12T07:20:34.328 INFO:teuthology.orchestra.run.smithi046.stdout:3578 writesame 0x700d0f3 thru 0x700f27a data_size 0x1d(0x2188 bytes) 2024-09-12T07:20:34.331 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.331 INFO:teuthology.orchestra.run.smithi046.stdout:3580 write 0x9da3a41 thru 0x9db27d9 (0xed99 bytes) 2024-09-12T07:20:34.351 INFO:teuthology.orchestra.run.smithi046.stdout:3581 trunc from 0xe55b97a to 0x263691a 2024-09-12T07:20:34.351 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:34.351 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:34.364 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:34.392 INFO:teuthology.orchestra.run.smithi046.stdout:3582 write 0xcf6c956 thru 0xcf7bd21 (0xf3cc bytes) 2024-09-12T07:20:34.397 INFO:teuthology.orchestra.run.smithi046.stdout:3583 punch from 0x4693d3c to 0x4695a69, (0x1d2d bytes) 2024-09-12T07:20:34.397 INFO:teuthology.orchestra.run.smithi046.stdout:3584 write 0x8acbd52 thru 0x8ad5743 (0x99f2 bytes) 2024-09-12T07:20:34.402 INFO:teuthology.orchestra.run.smithi046.stdout:3585 punch from 0x2c06e8b to 0x2c0ecd6, (0x7e4b bytes) 2024-09-12T07:20:34.402 INFO:teuthology.orchestra.run.smithi046.stdout:3586 read 0x61e3715 thru 0x61f32e1 (0xfbcd bytes) 2024-09-12T07:20:34.402 INFO:teuthology.orchestra.run.smithi046.stdout:3587 read 0xae60f03 thru 0xae65a2d (0x4b2b bytes) 2024-09-12T07:20:34.402 INFO:teuthology.orchestra.run.smithi046.stdout:3588 write 0x3929cfe thru 0x392f54f (0x5852 bytes) 2024-09-12T07:20:34.404 INFO:teuthology.orchestra.run.smithi046.stdout:3589 writesame 0xc982ac3 thru 0xc98deb8 data_size 0x55(0xb3f6 bytes) 2024-09-12T07:20:34.409 INFO:teuthology.orchestra.run.smithi046.stdout:3590 punch from 0x1dc68a3 to 0x1dcfd2f, (0x948c bytes) 2024-09-12T07:20:34.409 INFO:teuthology.orchestra.run.smithi046.stdout:3591 writesame 0xb37a817 thru 0xb38409f data_size 0x1(0x9889 bytes) 2024-09-12T07:20:34.416 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:34.418+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:34.433 INFO:teuthology.orchestra.run.smithi046.stdout:3592 write 0xe563b4d thru 0xe563bf5 (0xa9 bytes) 2024-09-12T07:20:34.436 INFO:teuthology.orchestra.run.smithi046.stdout:3594 write 0x77a961 thru 0x78582b (0xaecb bytes) 2024-09-12T07:20:34.438 INFO:teuthology.orchestra.run.smithi046.stdout:3595 write 0xb5d70cb thru 0xb5da828 (0x375e bytes) 2024-09-12T07:20:34.443 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.443 INFO:teuthology.orchestra.run.smithi046.stdout:3598 writesame 0x1b54c9a thru 0x1b60d43 data_size 0x5b(0xc0aa bytes) 2024-09-12T07:20:34.445 INFO:teuthology.orchestra.run.smithi046.stdout:3599 write 0xceed4d thru 0xcfcde3 (0xe097 bytes) 2024-09-12T07:20:34.449 INFO:teuthology.orchestra.run.smithi046.stdout:3600 writesame 0x849b78c thru 0x84a8843 data_size 0x4(0xd0b8 bytes) 2024-09-12T07:20:34.477 INFO:teuthology.orchestra.run.smithi046.stdout:3601 trunc from 0xe563bf6 to 0x10172f9 2024-09-12T07:20:34.499 INFO:teuthology.orchestra.run.smithi046.stdout:3602 punch from 0x449022 to 0x44db34, (0x4b12 bytes) 2024-09-12T07:20:34.515 INFO:teuthology.orchestra.run.smithi046.stdout:3603 write 0x9b0242a thru 0x9b05fc8 (0x3b9f bytes) 2024-09-12T07:20:34.516 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:34.518+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:34.528 INFO:teuthology.orchestra.run.smithi046.stdout:3604 write 0xc72c301 thru 0xc7308f8 (0x45f8 bytes) 2024-09-12T07:20:34.530 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:34.530 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:34.532 INFO:teuthology.orchestra.run.smithi046.stdout:3605 write 0xed822 thru 0xf0aa1 (0x3280 bytes) 2024-09-12T07:20:34.534 INFO:teuthology.orchestra.run.smithi046.stdout:3607 writesame 0x4eb6f73 thru 0x4ebdc40 data_size 0x13(0x6cce bytes) 2024-09-12T07:20:34.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:34.542 INFO:teuthology.orchestra.run.smithi046.stdout:3608 writesame 0x4e2006e thru 0x4e2cbe6 data_size 0x3(0xcb79 bytes) 2024-09-12T07:20:34.550 INFO:teuthology.orchestra.run.smithi046.stdout:3610 write 0xc17133d thru 0xc17d83c (0xc500 bytes) 2024-09-12T07:20:34.566 INFO:teuthology.orchestra.run.smithi046.stdout:3612 trunc from 0xc7308f9 to 0x2ad9c50 2024-09-12T07:20:34.579 INFO:teuthology.orchestra.run.smithi046.stdout:3613 read 0x1df56c7 thru 0x1dfd63b (0x7f75 bytes) 2024-09-12T07:20:34.591 INFO:teuthology.orchestra.run.smithi046.stdout:3614 write 0x767858b thru 0x767e98f (0x6405 bytes) 2024-09-12T07:20:34.595 INFO:teuthology.orchestra.run.smithi046.stdout:3615 read 0x5fd6de6 thru 0x5fdc7a4 (0x59bf bytes) 2024-09-12T07:20:34.595 INFO:teuthology.orchestra.run.smithi046.stdout:3616 read 0xf860af thru 0xf8ec38 (0x8b8a bytes) 2024-09-12T07:20:34.595 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.596 INFO:teuthology.orchestra.run.smithi046.stdout:3618 punch from 0x47ed4a8 to 0x47f20d5, (0x4c2d bytes) 2024-09-12T07:20:34.596 INFO:teuthology.orchestra.run.smithi046.stdout:3619 write 0x66f5945 thru 0x6705384 (0xfa40 bytes) 2024-09-12T07:20:34.600 INFO:teuthology.orchestra.run.smithi046.stdout:3620 read 0x8cba77 thru 0x8d6794 (0xad1e bytes) 2024-09-12T07:20:34.601 INFO:teuthology.orchestra.run.smithi046.stdout:3621 read 0x289520a thru 0x28a099e (0xb795 bytes) 2024-09-12T07:20:34.601 INFO:teuthology.orchestra.run.smithi046.stdout:3622 write 0x5c8ba05 thru 0x5c974b5 (0xbab1 bytes) 2024-09-12T07:20:34.606 INFO:teuthology.orchestra.run.smithi046.stdout:3624 write 0x5ee916f thru 0x5eec486 (0x3318 bytes) 2024-09-12T07:20:34.614 INFO:teuthology.orchestra.run.smithi046.stdout:3625 trunc from 0x767e990 to 0xc310b97 2024-09-12T07:20:34.617 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:34.619+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:34.620 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.620 INFO:teuthology.orchestra.run.smithi046.stdout:3628 write 0x9b2fe01 thru 0x9b37c03 (0x7e03 bytes) 2024-09-12T07:20:34.625 INFO:teuthology.orchestra.run.smithi046.stdout:3629 write 0x1014b55 thru 0x10216bc (0xcb68 bytes) 2024-09-12T07:20:34.627 INFO:teuthology.orchestra.run.smithi046.stdout:3632 write 0x8f803e7 thru 0x8f8c974 (0xc58e bytes) 2024-09-12T07:20:34.631 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.632 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.640 INFO:teuthology.orchestra.run.smithi046.stdout:3636 write 0xe1ad423 thru 0xe1b9a99 (0xc677 bytes) 2024-09-12T07:20:34.646 INFO:teuthology.orchestra.run.smithi046.stdout:3637 read 0xa605e38 thru 0xa614f68 (0xf131 bytes) 2024-09-12T07:20:34.646 INFO:teuthology.orchestra.run.smithi046.stdout:3638 write 0x2c91b26 thru 0x2c936df (0x1bba bytes) 2024-09-12T07:20:34.649 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.649 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.657 INFO:teuthology.orchestra.run.smithi046.stdout:3642 trunc from 0xe1b9a9a to 0x75caea0 2024-09-12T07:20:34.673 INFO:teuthology.orchestra.run.smithi046.stdout:3644 read 0x4fc1aa5 thru 0x4fd18d5 (0xfe31 bytes) 2024-09-12T07:20:34.685 INFO:teuthology.orchestra.run.smithi046.stdout:3647 write 0xb4cc65a thru 0xb4cd93b (0x12e2 bytes) 2024-09-12T07:20:34.688 INFO:teuthology.orchestra.run.smithi046.stdout:3649 punch from 0x24f7cd0 to 0x2501ccd, (0x9ffd bytes) 2024-09-12T07:20:34.688 INFO:teuthology.orchestra.run.smithi046.stdout:3650 punch from 0xf6f4f5 to 0xf725ee, (0x30f9 bytes) 2024-09-12T07:20:34.689 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:34.689 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:34.689 INFO:teuthology.orchestra.run.smithi046.stdout:3651 write 0xa9f3735 thru 0xaa005f3 (0xcebf bytes) 2024-09-12T07:20:34.694 INFO:teuthology.orchestra.run.smithi046.stdout:3652 writesame 0x1320bec thru 0x132773a data_size 0x1(0x6b4f bytes) 2024-09-12T07:20:34.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:34.706 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:20:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:20:34.715 INFO:teuthology.orchestra.run.smithi046.stdout:3653 trunc from 0xb4cd93c to 0x3da4b29 2024-09-12T07:20:34.718 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:34.719+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:34.725 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-12T07:20:34.311899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":949127292,"num_objects":1226,"num_object_clones":296,"num_object_copies":2452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1226,"num_whiteouts":564,"num_read":164868,"num_read_kb":772868,"num_write":10422,"num_write_kb":413167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":49942512,"num_keys_recovered":48,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8516,"ondisk_log_size":8516,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":860872,"kb_used_data":559928,"kb_used_omap":12,"kb_used_meta":300659,"kb_avail":754113848,"statfs":{"total":773094113280,"available":772212580352,"internally_reserved":0,"allocated":573366272,"data_stored":676206150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":307875377},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":64279114,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":18768,"num_read_kb":122376,"num_write":2338,"num_write_kb":38842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017514"},"pg_stats":[{"pgid":"3.1f","version":"68'337","reported_seq":2020,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.304173+0000","last_change":"2024-09-12T07:19:44.455827+0000","last_active":"2024-09-12T07:20:32.304173+0000","last_peered":"2024-09-12T07:20:32.304173+0000","last_clean":"2024-09-12T07:20:32.304173+0000","last_became_active":"2024-09-12T07:19:42.451212+0000","last_became_peered":"2024-09-12T07:19:42.451212+0000","last_unstale":"2024-09-12T07:20:32.304173+0000","last_undegraded":"2024-09-12T07:20:32.304173+0000","last_fullsized":"2024-09-12T07:20:32.304173+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:04:31.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001023672,"stat_sum":{"num_bytes":13266909,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":3201,"num_read_kb":15912,"num_write":466,"num_write_kb":5844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"70'211","reported_seq":472,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.310404+0000","last_change":"2024-09-12T07:20:07.305432+0000","last_active":"2024-09-12T07:20:32.310404+0000","last_peered":"2024-09-12T07:20:32.310404+0000","last_clean":"2024-09-12T07:20:32.310404+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:20:32.310404+0000","last_undegraded":"2024-09-12T07:20:32.310404+0000","last_fullsized":"2024-09-12T07:20:32.310404+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:47.485606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081935099999999996,"stat_sum":{"num_bytes":45947739,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":17302,"num_read_kb":21358,"num_write":193,"num_write_kb":18619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"68'150","reported_seq":18011,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.312504+0000","last_change":"2024-09-12T07:19:44.464376+0000","last_active":"2024-09-12T07:20:32.312504+0000","last_peered":"2024-09-12T07:20:32.312504+0000","last_clean":"2024-09-12T07:20:32.312504+0000","last_became_active":"2024-09-12T07:19:42.451973+0000","last_became_peered":"2024-09-12T07:19:42.451973+0000","last_unstale":"2024-09-12T07:20:32.312504+0000","last_undegraded":"2024-09-12T07:20:32.312504+0000","last_fullsized":"2024-09-12T07:20:32.312504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:30:52.131510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036949999999999,"stat_sum":{"num_bytes":28202558,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":21,"num_read":17798,"num_read_kb":28430,"num_write":130,"num_write_kb":11932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"71'288","reported_seq":2107,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.476074+0000","last_change":"2024-09-12T07:19:44.464287+0000","last_active":"2024-09-12T07:20:32.476074+0000","last_peered":"2024-09-12T07:20:32.476074+0000","last_clean":"2024-09-12T07:20:32.476074+0000","last_became_active":"2024-09-12T07:19:42.455338+0000","last_became_peered":"2024-09-12T07:19:42.455338+0000","last_unstale":"2024-09-12T07:20:32.476074+0000","last_undegraded":"2024-09-12T07:20:32.476074+0000","last_fullsized":"2024-09-12T07:20:32.476074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:41.038321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012347180000000001,"stat_sum":{"num_bytes":26997538,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":2318,"num_read_kb":11364,"num_write":364,"num_write_kb":4866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"70'266","reported_seq":17721,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.533800+0000","last_change":"2024-09-12T07:19:44.456652+0000","last_active":"2024-09-12T07:20:32.533800+0000","last_peered":"2024-09-12T07:20:32.533800+0000","last_clean":"2024-09-12T07:20:32.533800+0000","last_became_active":"2024-09-12T07:19:42.454990+0000","last_became_peered":"2024-09-12T07:19:42.454990+0000","last_unstale":"2024-09-12T07:20:32.533800+0000","last_undegraded":"2024-09-12T07:20:32.533800+0000","last_fullsized":"2024-09-12T07:20:32.533800+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:29:56.750843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049866,"stat_sum":{"num_bytes":31412648,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":18271,"num_read_kb":45982,"num_write":280,"num_write_kb":14039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"71'242","reported_seq":2478,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.475673+0000","last_change":"2024-09-12T07:19:44.459417+0000","last_active":"2024-09-12T07:20:32.475673+0000","last_peered":"2024-09-12T07:20:32.475673+0000","last_clean":"2024-09-12T07:20:32.475673+0000","last_became_active":"2024-09-12T07:19:42.454620+0000","last_became_peered":"2024-09-12T07:19:42.454620+0000","last_unstale":"2024-09-12T07:20:32.475673+0000","last_undegraded":"2024-09-12T07:20:32.475673+0000","last_fullsized":"2024-09-12T07:20:32.475673+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:45:49.306030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090501500000000001,"stat_sum":{"num_bytes":15071959,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":22,"num_read":2993,"num_read_kb":9861,"num_write":230,"num_write_kb":14065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"68'299","reported_seq":4891,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.480530+0000","last_change":"2024-09-12T07:20:10.196744+0000","last_active":"2024-09-12T07:20:32.480530+0000","last_peered":"2024-09-12T07:20:32.480530+0000","last_clean":"2024-09-12T07:20:32.480530+0000","last_became_active":"2024-09-12T07:20:10.195312+0000","last_became_peered":"2024-09-12T07:20:10.195312+0000","last_unstale":"2024-09-12T07:20:32.480530+0000","last_undegraded":"2024-09-12T07:20:32.480530+0000","last_fullsized":"2024-09-12T07:20:32.480530+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:36.757545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113156,"stat_sum":{"num_bytes":27067089,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":5263,"num_read_kb":38807,"num_write":401,"num_write_kb":11737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1101525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"71'310","reported_seq":2142,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.460018+0000","last_change":"2024-09-12T07:20:07.446518+0000","last_active":"2024-09-12T07:20:32.460018+0000","last_peered":"2024-09-12T07:20:32.460018+0000","last_clean":"2024-09-12T07:20:32.460018+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:20:32.460018+0000","last_undegraded":"2024-09-12T07:20:32.460018+0000","last_fullsized":"2024-09-12T07:20:32.460018+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:40:40.304122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000443294,"stat_sum":{"num_bytes":40263224,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":4439,"num_read_kb":36222,"num_write":405,"num_write_kb":14391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"71'254","reported_seq":4440,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.533592+0000","last_change":"2024-09-12T07:19:44.456115+0000","last_active":"2024-09-12T07:20:32.533592+0000","last_peered":"2024-09-12T07:20:32.533592+0000","last_clean":"2024-09-12T07:20:32.533592+0000","last_became_active":"2024-09-12T07:19:42.452617+0000","last_became_peered":"2024-09-12T07:19:42.452617+0000","last_unstale":"2024-09-12T07:20:32.533592+0000","last_undegraded":"2024-09-12T07:20:32.533592+0000","last_fullsized":"2024-09-12T07:20:32.533592+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:31:01.422030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077251,"stat_sum":{"num_bytes":26742248,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":5357,"num_read_kb":12982,"num_write":236,"num_write_kb":20347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"68'122","reported_seq":400,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.312682+0000","last_change":"2024-09-12T07:19:44.463507+0000","last_active":"2024-09-12T07:20:32.312682+0000","last_peered":"2024-09-12T07:20:32.312682+0000","last_clean":"2024-09-12T07:20:32.312682+0000","last_became_active":"2024-09-12T07:19:42.455210+0000","last_became_peered":"2024-09-12T07:19:42.455210+0000","last_unstale":"2024-09-12T07:20:32.312682+0000","last_undegraded":"2024-09-12T07:20:32.312682+0000","last_fullsized":"2024-09-12T07:20:32.312682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:33.710716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049753700000000004,"stat_sum":{"num_bytes":14035109,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":20,"num_read":186,"num_read_kb":4520,"num_write":100,"num_write_kb":5250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"68'146","reported_seq":5930,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.536890+0000","last_change":"2024-09-12T07:20:07.261139+0000","last_active":"2024-09-12T07:20:32.536890+0000","last_peered":"2024-09-12T07:20:32.536890+0000","last_clean":"2024-09-12T07:20:32.536890+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:20:32.536890+0000","last_undegraded":"2024-09-12T07:20:32.536890+0000","last_fullsized":"2024-09-12T07:20:32.536890+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:52.445732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012225490000000001,"stat_sum":{"num_bytes":21134923,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":5928,"num_read_kb":41383,"num_write":127,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"68'206","reported_seq":4275,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.536650+0000","last_change":"2024-09-12T07:19:44.481796+0000","last_active":"2024-09-12T07:20:32.536650+0000","last_peered":"2024-09-12T07:20:32.536650+0000","last_clean":"2024-09-12T07:20:32.536650+0000","last_became_active":"2024-09-12T07:19:42.452277+0000","last_became_peered":"2024-09-12T07:19:42.452277+0000","last_unstale":"2024-09-12T07:20:32.536650+0000","last_undegraded":"2024-09-12T07:20:32.536650+0000","last_fullsized":"2024-09-12T07:20:32.536650+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:15:19.816581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012485160000000001,"stat_sum":{"num_bytes":33238886,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":4450,"num_read_kb":50918,"num_write":205,"num_write_kb":26359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":118,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.350602+0000","last_change":"2024-09-12T07:19:32.474512+0000","last_active":"2024-09-12T07:20:32.350602+0000","last_peered":"2024-09-12T07:20:32.350602+0000","last_clean":"2024-09-12T07:20:32.350602+0000","last_became_active":"2024-09-12T07:19:30.219990+0000","last_became_peered":"2024-09-12T07:19:30.219990+0000","last_unstale":"2024-09-12T07:20:32.350602+0000","last_undegraded":"2024-09-12T07:20:32.350602+0000","last_fullsized":"2024-09-12T07:20:32.350602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:32.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046339800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"68'319","reported_seq":1401,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.533039+0000","last_change":"2024-09-12T07:19:44.455257+0000","last_active":"2024-09-12T07:20:32.533039+0000","last_peered":"2024-09-12T07:20:32.533039+0000","last_clean":"2024-09-12T07:20:32.533039+0000","last_became_active":"2024-09-12T07:19:42.449730+0000","last_became_peered":"2024-09-12T07:19:42.449730+0000","last_unstale":"2024-09-12T07:20:32.533039+0000","last_undegraded":"2024-09-12T07:20:32.533039+0000","last_fullsized":"2024-09-12T07:20:32.533039+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:51:32.774607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059259600000000003,"stat_sum":{"num_bytes":18077355,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":22,"num_read":2062,"num_read_kb":9229,"num_write":399,"num_write_kb":8485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.306562+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:20:32.306562+0000","last_peered":"2024-09-12T07:20:32.306562+0000","last_clean":"2024-09-12T07:20:32.306562+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:20:32.306562+0000","last_undegraded":"2024-09-12T07:20:32.306562+0000","last_fullsized":"2024-09-12T07:20:32.306562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"70'281","reported_seq":993,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.310206+0000","last_change":"2024-09-12T07:20:07.211023+0000","last_active":"2024-09-12T07:20:32.310206+0000","last_peered":"2024-09-12T07:20:32.310206+0000","last_clean":"2024-09-12T07:20:32.310206+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:20:32.310206+0000","last_undegraded":"2024-09-12T07:20:32.310206+0000","last_fullsized":"2024-09-12T07:20:32.310206+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:11:36.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087343100000000003,"stat_sum":{"num_bytes":22892712,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":1462,"num_read_kb":12438,"num_write":364,"num_write_kb":5088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":118,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.333316+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:20:32.333316+0000","last_peered":"2024-09-12T07:20:32.333316+0000","last_clean":"2024-09-12T07:20:32.333316+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:20:32.333316+0000","last_undegraded":"2024-09-12T07:20:32.333316+0000","last_fullsized":"2024-09-12T07:20:32.333316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"68'247","reported_seq":950,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.531686+0000","last_change":"2024-09-12T07:19:44.458925+0000","last_active":"2024-09-12T07:20:32.531686+0000","last_peered":"2024-09-12T07:20:32.531686+0000","last_clean":"2024-09-12T07:20:32.531686+0000","last_became_active":"2024-09-12T07:19:42.452704+0000","last_became_peered":"2024-09-12T07:19:42.452704+0000","last_unstale":"2024-09-12T07:20:32.531686+0000","last_undegraded":"2024-09-12T07:20:32.531686+0000","last_fullsized":"2024-09-12T07:20:32.531686+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:16.578407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045577200000000001,"stat_sum":{"num_bytes":21841828,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":1516,"num_read_kb":4054,"num_write":313,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.306574+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:20:32.306574+0000","last_peered":"2024-09-12T07:20:32.306574+0000","last_clean":"2024-09-12T07:20:32.306574+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:20:32.306574+0000","last_undegraded":"2024-09-12T07:20:32.306574+0000","last_fullsized":"2024-09-12T07:20:32.306574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"68'220","reported_seq":4164,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.312584+0000","last_change":"2024-09-12T07:20:07.289858+0000","last_active":"2024-09-12T07:20:32.312584+0000","last_peered":"2024-09-12T07:20:32.312584+0000","last_clean":"2024-09-12T07:20:32.312584+0000","last_became_active":"2024-09-12T07:20:07.183819+0000","last_became_peered":"2024-09-12T07:20:07.183819+0000","last_unstale":"2024-09-12T07:20:32.312584+0000","last_undegraded":"2024-09-12T07:20:32.312584+0000","last_fullsized":"2024-09-12T07:20:32.312584+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:26.537525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015134619999999999,"stat_sum":{"num_bytes":45604391,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":4626,"num_read_kb":44381,"num_write":258,"num_write_kb":18869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":3070354,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"68'149","reported_seq":1611,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.533620+0000","last_change":"2024-09-12T07:19:44.456465+0000","last_active":"2024-09-12T07:20:32.533620+0000","last_peered":"2024-09-12T07:20:32.533620+0000","last_clean":"2024-09-12T07:20:32.533620+0000","last_became_active":"2024-09-12T07:19:42.454103+0000","last_became_peered":"2024-09-12T07:19:42.454103+0000","last_unstale":"2024-09-12T07:20:32.533620+0000","last_undegraded":"2024-09-12T07:20:32.533620+0000","last_fullsized":"2024-09-12T07:20:32.533620+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:15.119417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010980880000000001,"stat_sum":{"num_bytes":18636911,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":24,"num_read":1378,"num_read_kb":3209,"num_write":127,"num_write_kb":4808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":120,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.312367+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:20:32.312367+0000","last_peered":"2024-09-12T07:20:32.312367+0000","last_clean":"2024-09-12T07:20:32.312367+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:20:32.312367+0000","last_undegraded":"2024-09-12T07:20:32.312367+0000","last_fullsized":"2024-09-12T07:20:32.312367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"68'271","reported_seq":2000,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.306616+0000","last_change":"2024-09-12T07:19:44.458868+0000","last_active":"2024-09-12T07:20:32.306616+0000","last_peered":"2024-09-12T07:20:32.306616+0000","last_clean":"2024-09-12T07:20:32.306616+0000","last_became_active":"2024-09-12T07:19:42.451807+0000","last_became_peered":"2024-09-12T07:19:42.451807+0000","last_unstale":"2024-09-12T07:20:32.306616+0000","last_undegraded":"2024-09-12T07:20:32.306616+0000","last_fullsized":"2024-09-12T07:20:32.306616+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:41:18.028635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048363399999999998,"stat_sum":{"num_bytes":29481674,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":25,"num_read":2960,"num_read_kb":8628,"num_write":336,"num_write_kb":9392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":130,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.447351+0000","last_change":"2024-09-12T07:20:07.183543+0000","last_active":"2024-09-12T07:20:32.447351+0000","last_peered":"2024-09-12T07:20:32.447351+0000","last_clean":"2024-09-12T07:20:32.447351+0000","last_became_active":"2024-09-12T07:20:07.183213+0000","last_became_peered":"2024-09-12T07:20:07.183213+0000","last_unstale":"2024-09-12T07:20:32.447351+0000","last_undegraded":"2024-09-12T07:20:32.447351+0000","last_fullsized":"2024-09-12T07:20:32.447351+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.634459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"68'144","reported_seq":2853,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.303766+0000","last_change":"2024-09-12T07:19:44.455560+0000","last_active":"2024-09-12T07:20:32.303766+0000","last_peered":"2024-09-12T07:20:32.303766+0000","last_clean":"2024-09-12T07:20:32.303766+0000","last_became_active":"2024-09-12T07:19:42.451970+0000","last_became_peered":"2024-09-12T07:19:42.451970+0000","last_unstale":"2024-09-12T07:20:32.303766+0000","last_undegraded":"2024-09-12T07:20:32.303766+0000","last_fullsized":"2024-09-12T07:20:32.303766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:05:25.259536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083352100000000004,"stat_sum":{"num_bytes":43142164,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":3547,"num_read_kb":33681,"num_write":158,"num_write_kb":23103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":118,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.332995+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:20:32.332995+0000","last_peered":"2024-09-12T07:20:32.332995+0000","last_clean":"2024-09-12T07:20:32.332995+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:20:32.332995+0000","last_undegraded":"2024-09-12T07:20:32.332995+0000","last_fullsized":"2024-09-12T07:20:32.332995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"68'688","reported_seq":3038,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.310137+0000","last_change":"2024-09-12T07:20:07.467030+0000","last_active":"2024-09-12T07:20:32.310137+0000","last_peered":"2024-09-12T07:20:32.310137+0000","last_clean":"2024-09-12T07:20:32.310137+0000","last_became_active":"2024-09-12T07:20:07.180577+0000","last_became_peered":"2024-09-12T07:20:07.180577+0000","last_unstale":"2024-09-12T07:20:32.310137+0000","last_undegraded":"2024-09-12T07:20:32.310137+0000","last_fullsized":"2024-09-12T07:20:32.310137+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:24:11.168066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089638599999999999,"stat_sum":{"num_bytes":32630331,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":22,"num_read":5273,"num_read_kb":33681,"num_write":1110,"num_write_kb":14584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":261251,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":119,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.312384+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:20:32.312384+0000","last_peered":"2024-09-12T07:20:32.312384+0000","last_clean":"2024-09-12T07:20:32.312384+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:20:32.312384+0000","last_undegraded":"2024-09-12T07:20:32.312384+0000","last_fullsized":"2024-09-12T07:20:32.312384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":209,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.333010+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:20:32.333010+0000","last_peered":"2024-09-12T07:20:32.333010+0000","last_clean":"2024-09-12T07:20:32.333010+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:20:32.333010+0000","last_undegraded":"2024-09-12T07:20:32.333010+0000","last_fullsized":"2024-09-12T07:20:32.333010+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"71'756","reported_seq":1516,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.477577+0000","last_change":"2024-09-12T07:20:07.428482+0000","last_active":"2024-09-12T07:20:32.477577+0000","last_peered":"2024-09-12T07:20:32.477577+0000","last_clean":"2024-09-12T07:20:32.477577+0000","last_became_active":"2024-09-12T07:20:07.180763+0000","last_became_peered":"2024-09-12T07:20:07.180763+0000","last_unstale":"2024-09-12T07:20:32.477577+0000","last_undegraded":"2024-09-12T07:20:32.477577+0000","last_fullsized":"2024-09-12T07:20:32.477577+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:02.054433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047491800000000001,"stat_sum":{"num_bytes":33165368,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":4144,"num_read_kb":20152,"num_write":1269,"num_write_kb":18039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":8633990,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"68'314","reported_seq":3447,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.447475+0000","last_change":"2024-09-12T07:19:44.469508+0000","last_active":"2024-09-12T07:20:32.447475+0000","last_peered":"2024-09-12T07:20:32.447475+0000","last_clean":"2024-09-12T07:20:32.447475+0000","last_became_active":"2024-09-12T07:19:42.450230+0000","last_became_peered":"2024-09-12T07:19:42.450230+0000","last_unstale":"2024-09-12T07:20:32.447475+0000","last_undegraded":"2024-09-12T07:20:32.447475+0000","last_fullsized":"2024-09-12T07:20:32.447475+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:34:50.649525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076460899999999997,"stat_sum":{"num_bytes":31927129,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":23,"num_read":5084,"num_read_kb":25900,"num_write":314,"num_write_kb":3106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"71'183","reported_seq":3236,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.350767+0000","last_change":"2024-09-12T07:19:44.480981+0000","last_active":"2024-09-12T07:20:32.350767+0000","last_peered":"2024-09-12T07:20:32.350767+0000","last_clean":"2024-09-12T07:20:32.350767+0000","last_became_active":"2024-09-12T07:19:42.452637+0000","last_became_peered":"2024-09-12T07:19:42.452637+0000","last_unstale":"2024-09-12T07:20:32.350767+0000","last_undegraded":"2024-09-12T07:20:32.350767+0000","last_fullsized":"2024-09-12T07:20:32.350767+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:10.659877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000680892,"stat_sum":{"num_bytes":35360399,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":25,"num_read":3083,"num_read_kb":31017,"num_write":197,"num_write_kb":13093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"70'219","reported_seq":1293,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.310327+0000","last_change":"2024-09-12T07:20:07.379907+0000","last_active":"2024-09-12T07:20:32.310327+0000","last_peered":"2024-09-12T07:20:32.310327+0000","last_clean":"2024-09-12T07:20:32.310327+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:20:32.310327+0000","last_undegraded":"2024-09-12T07:20:32.310327+0000","last_fullsized":"2024-09-12T07:20:32.310327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:35.725520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054644100000000005,"stat_sum":{"num_bytes":28454093,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":20,"num_read":1991,"num_read_kb":11621,"num_write":221,"num_write_kb":13079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"70'225","reported_seq":7468,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.531224+0000","last_change":"2024-09-12T07:20:07.249347+0000","last_active":"2024-09-12T07:20:32.531224+0000","last_peered":"2024-09-12T07:20:32.531224+0000","last_clean":"2024-09-12T07:20:32.531224+0000","last_became_active":"2024-09-12T07:20:07.183065+0000","last_became_peered":"2024-09-12T07:20:07.183065+0000","last_unstale":"2024-09-12T07:20:32.531224+0000","last_undegraded":"2024-09-12T07:20:32.531224+0000","last_fullsized":"2024-09-12T07:20:32.531224+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:47.361795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050446300000000003,"stat_sum":{"num_bytes":39840262,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":7879,"num_read_kb":42906,"num_write":199,"num_write_kb":14091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":8578058,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"68'159","reported_seq":1718,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.333251+0000","last_change":"2024-09-12T07:19:44.482470+0000","last_active":"2024-09-12T07:20:32.333251+0000","last_peered":"2024-09-12T07:20:32.333251+0000","last_clean":"2024-09-12T07:20:32.333251+0000","last_became_active":"2024-09-12T07:19:42.450577+0000","last_became_peered":"2024-09-12T07:19:42.450577+0000","last_unstale":"2024-09-12T07:20:32.333251+0000","last_undegraded":"2024-09-12T07:20:32.333251+0000","last_fullsized":"2024-09-12T07:20:32.333251+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:44:46.567235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317836,"stat_sum":{"num_bytes":34488353,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":1814,"num_read_kb":19313,"num_write":175,"num_write_kb":8379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"71'326","reported_seq":3745,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.532247+0000","last_change":"2024-09-12T07:19:44.481850+0000","last_active":"2024-09-12T07:20:32.532247+0000","last_peered":"2024-09-12T07:20:32.532247+0000","last_clean":"2024-09-12T07:20:32.532247+0000","last_became_active":"2024-09-12T07:19:42.452236+0000","last_became_peered":"2024-09-12T07:19:42.452236+0000","last_unstale":"2024-09-12T07:20:32.532247+0000","last_undegraded":"2024-09-12T07:20:32.532247+0000","last_fullsized":"2024-09-12T07:20:32.532247+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:44:50.439574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012939710000000001,"stat_sum":{"num_bytes":21428311,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":5182,"num_read_kb":23521,"num_write":424,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"68'319","reported_seq":1026,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.536511+0000","last_change":"2024-09-12T07:19:44.482179+0000","last_active":"2024-09-12T07:20:32.536511+0000","last_peered":"2024-09-12T07:20:32.536511+0000","last_clean":"2024-09-12T07:20:32.536511+0000","last_became_active":"2024-09-12T07:19:42.452155+0000","last_became_peered":"2024-09-12T07:19:42.452155+0000","last_unstale":"2024-09-12T07:20:32.536511+0000","last_undegraded":"2024-09-12T07:20:32.536511+0000","last_fullsized":"2024-09-12T07:20:32.536511+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:06:49.990730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011884759999999999,"stat_sum":{"num_bytes":15047103,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":1807,"num_read_kb":7338,"num_write":397,"num_write_kb":14969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"70'187","reported_seq":657,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.532297+0000","last_change":"2024-09-12T07:19:44.481020+0000","last_active":"2024-09-12T07:20:32.532297+0000","last_peered":"2024-09-12T07:20:32.532297+0000","last_clean":"2024-09-12T07:20:32.532297+0000","last_became_active":"2024-09-12T07:19:42.450535+0000","last_became_peered":"2024-09-12T07:19:42.450535+0000","last_unstale":"2024-09-12T07:20:32.532297+0000","last_undegraded":"2024-09-12T07:20:32.532297+0000","last_fullsized":"2024-09-12T07:20:32.532297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:54.851331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073032100000000003,"stat_sum":{"num_bytes":43465959,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":719,"num_read_kb":6958,"num_write":212,"num_write_kb":16344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"68'165","reported_seq":2759,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.531426+0000","last_change":"2024-09-12T07:19:44.481275+0000","last_active":"2024-09-12T07:20:32.531426+0000","last_peered":"2024-09-12T07:20:32.531426+0000","last_clean":"2024-09-12T07:20:32.531426+0000","last_became_active":"2024-09-12T07:19:42.452104+0000","last_became_peered":"2024-09-12T07:19:42.452104+0000","last_unstale":"2024-09-12T07:20:32.531426+0000","last_undegraded":"2024-09-12T07:20:32.531426+0000","last_fullsized":"2024-09-12T07:20:32.531426+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:55:17.910307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079083699999999996,"stat_sum":{"num_bytes":18167110,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":2973,"num_read_kb":30025,"num_write":156,"num_write_kb":13865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"68'123","reported_seq":1571,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.303769+0000","last_change":"2024-09-12T07:19:44.456086+0000","last_active":"2024-09-12T07:20:32.303769+0000","last_peered":"2024-09-12T07:20:32.303769+0000","last_clean":"2024-09-12T07:20:32.303769+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:20:32.303769+0000","last_undegraded":"2024-09-12T07:20:32.303769+0000","last_fullsized":"2024-09-12T07:20:32.303769+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:09.412505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060746,"stat_sum":{"num_bytes":30934923,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":12,"num_read":1342,"num_read_kb":17954,"num_write":159,"num_write_kb":9424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"70'315","reported_seq":18272,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.536282+0000","last_change":"2024-09-12T07:19:44.482564+0000","last_active":"2024-09-12T07:20:32.536282+0000","last_peered":"2024-09-12T07:20:32.536282+0000","last_clean":"2024-09-12T07:20:32.536282+0000","last_became_active":"2024-09-12T07:19:42.456127+0000","last_became_peered":"2024-09-12T07:19:42.456127+0000","last_unstale":"2024-09-12T07:20:32.536282+0000","last_undegraded":"2024-09-12T07:20:32.536282+0000","last_fullsized":"2024-09-12T07:20:32.536282+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:48:43.140119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793039999999999,"stat_sum":{"num_bytes":60569699,"num_objects":51,"num_object_clones":16,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":18454,"num_read_kb":69068,"num_write":394,"num_write_kb":33928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":948536905,"num_objects":1222,"num_object_clones":296,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":564,"num_read":164802,"num_read_kb":772813,"num_write":10319,"num_write_kb":411537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":49942512,"num_keys_recovered":48,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565641216,"data_stored":672083868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8441,"ondisk_log_size":8441,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":128180,"kb_used_data":92832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94243660,"statfs":{"total":96636764160,"available":96505507840,"internally_reserved":0,"allocated":95059968,"data_stored":110884325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63352,"kb_used_data":32056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308488,"statfs":{"total":96636764160,"available":96571891712,"internally_reserved":0,"allocated":32825344,"data_stored":39034069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114300,"kb_used_data":78944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94257540,"statfs":{"total":96636764160,"available":96519720960,"internally_reserved":0,"allocated":80838656,"data_stored":95025823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179168,"kb_used_data":139660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94192672,"statfs":{"total":96636764160,"available":96453296128,"internally_reserved":0,"allocated":143011840,"data_stored":177291358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134004,"kb_used_data":88932,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94237836,"statfs":{"total":96636764160,"available":96499544064,"internally_reserved":0,"allocated":91066368,"data_stored":106265436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46692,"kb_used_data":11344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325148,"statfs":{"total":96636764160,"available":96588951552,"internally_reserved":0,"allocated":11616256,"data_stored":11092893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72332,"kb_used_data":32820,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94299508,"statfs":{"total":96636764160,"available":96562696192,"internally_reserved":0,"allocated":33607680,"data_stored":34627208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122844,"kb_used_data":83340,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94248996,"statfs":{"total":96636764160,"available":96510971904,"internally_reserved":0,"allocated":85340160,"data_stored":101985038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":83935232,"data_stored":101028548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32788480,"data_stored":34261020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10788864,"data_stored":10720573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":90251264,"data_stored":105899267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":142184448,"data_stored":176919038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":80027648,"data_stored":94659718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32010240,"data_stored":38667900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":93655040,"data_stored":109927804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:20:34.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:20:34.742 INFO:teuthology.orchestra.run.smithi046.stdout:3655 write 0x8709f41 thru 0x8713cf1 (0x9db1 bytes) 2024-09-12T07:20:34.758 INFO:teuthology.orchestra.run.smithi046.stdout:3656 write 0xdb356b6 thru 0xdb41693 (0xbfde bytes) 2024-09-12T07:20:34.763 INFO:teuthology.orchestra.run.smithi046.stdout:3657 read 0x8a522c thru 0x8b2b1a (0xd8ef bytes) 2024-09-12T07:20:34.764 INFO:teuthology.orchestra.run.smithi046.stdout:3658 read 0xc574464 thru 0xc577295 (0x2e32 bytes) 2024-09-12T07:20:34.764 INFO:teuthology.orchestra.run.smithi046.stdout:3659 write 0x99c0df0 thru 0x99c9c8e (0x8e9f bytes) 2024-09-12T07:20:34.767 INFO:teuthology.orchestra.run.smithi046.stdout:3660 punch from 0xaf0c5ba to 0xaf0f09d, (0x2ae3 bytes) 2024-09-12T07:20:34.768 INFO:teuthology.orchestra.run.smithi046.stdout:3661 read 0xbff3f91 thru 0xc000341 (0xc3b1 bytes) 2024-09-12T07:20:34.776 INFO:teuthology.orchestra.run.smithi046.stdout:3664 writesame 0xee58118 thru 0xee5c4bb data_size 0x4e(0x43a4 bytes) 2024-09-12T07:20:34.776 INFO:teuthology.orchestra.run.smithi046.stdout:3667 write 0x76ee24d thru 0x76f1dae (0x3b62 bytes) 2024-09-12T07:20:34.781 INFO:teuthology.orchestra.run.smithi046.stdout:3668 write 0xbd0fc52 thru 0xbd13c4f (0x3ffe bytes) 2024-09-12T07:20:34.785 INFO:teuthology.orchestra.run.smithi046.stdout:3669 write 0xeaea881 thru 0xeaf5a5d (0xb1dd bytes) 2024-09-12T07:20:34.789 INFO:teuthology.orchestra.run.smithi046.stdout:3671 punch from 0x99c5d86 to 0x99cd15c, (0x73d6 bytes) 2024-09-12T07:20:34.789 INFO:teuthology.orchestra.run.smithi046.stdout:3674 write 0x61e5151 thru 0x61e7b4c (0x29fc bytes) 2024-09-12T07:20:34.792 INFO:teuthology.orchestra.run.smithi046.stdout:3675 read 0x5b11ab2 thru 0x5b17407 (0x5956 bytes) 2024-09-12T07:20:34.792 INFO:teuthology.orchestra.run.smithi046.stdout:3677 write 0xd8ac62c thru 0xd8b709a (0xaa6f bytes) 2024-09-12T07:20:34.799 INFO:teuthology.orchestra.run.smithi046.stdout:3678 trunc from 0xee5c4bc to 0xb5e567b 2024-09-12T07:20:34.812 INFO:teuthology.orchestra.run.smithi046.stdout:3679 write 0x49aa19f thru 0x49b7415 (0xd277 bytes) 2024-09-12T07:20:34.816 INFO:teuthology.orchestra.run.smithi046.stdout:3681 read 0x2a5efcc thru 0x2a65610 (0x6645 bytes) 2024-09-12T07:20:34.819 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:34.821+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:34.824 INFO:teuthology.orchestra.run.smithi046.stdout:3682 write 0xd1de7fc thru 0xd1e4e8c (0x6691 bytes) 2024-09-12T07:20:34.829 INFO:teuthology.orchestra.run.smithi046.stdout:3684 write 0xbdf36ee thru 0xbdf4b55 (0x1468 bytes) 2024-09-12T07:20:34.831 INFO:teuthology.orchestra.run.smithi046.stdout:3685 write 0xa3aa71a thru 0xa3b74ae (0xcd95 bytes) 2024-09-12T07:20:34.835 INFO:teuthology.orchestra.run.smithi046.stdout:3686 read 0xb61d400 thru 0xb622878 (0x5479 bytes) 2024-09-12T07:20:34.835 INFO:teuthology.orchestra.run.smithi046.stdout:3687 read 0x861153d thru 0x8618455 (0x6f19 bytes) 2024-09-12T07:20:34.838 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:34.838 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:34.842 INFO:teuthology.orchestra.run.smithi046.stdout:3688 write 0xe9aa40a thru 0xe9afadc (0x56d3 bytes) 2024-09-12T07:20:34.845 INFO:teuthology.orchestra.run.smithi046.stdout:3690 read 0x1ea8379 thru 0x1eafec7 (0x7b4f bytes) 2024-09-12T07:20:34.846 INFO:teuthology.orchestra.run.smithi046.stdout:3692 writesame 0x1d45eef thru 0x1d4c435 data_size 0x1(0x6547 bytes) 2024-09-12T07:20:34.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:34.855 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.855 INFO:teuthology.orchestra.run.smithi046.stdout:3694 punch from 0x333c15c to 0x334617b, (0xa01f bytes) 2024-09-12T07:20:34.856 INFO:teuthology.orchestra.run.smithi046.stdout:3695 punch from 0x6278c38 to 0x628142d, (0x87f5 bytes) 2024-09-12T07:20:34.856 INFO:teuthology.orchestra.run.smithi046.stdout:3697 write 0xc0b1a80 thru 0xc0b3def (0x2370 bytes) 2024-09-12T07:20:34.858 INFO:teuthology.orchestra.run.smithi046.stdout:3699 read 0xc931ca1 thru 0xc9370c3 (0x5423 bytes) 2024-09-12T07:20:34.859 INFO:teuthology.orchestra.run.smithi046.stdout:3701 write 0x13c5cf8 thru 0x13c9db7 (0x40c0 bytes) 2024-09-12T07:20:34.871 INFO:teuthology.orchestra.run.smithi046.stdout:3703 trunc from 0xe9afadd to 0x53b40d2 2024-09-12T07:20:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:3704 write 0x2d09833 thru 0x2d12725 (0x8ef3 bytes) 2024-09-12T07:20:34.891 INFO:teuthology.orchestra.run.smithi046.stdout:3709 write 0xcfaa69 thru 0xd09ecc (0xf464 bytes) 2024-09-12T07:20:34.893 INFO:teuthology.orchestra.run.smithi046.stdout:3710 writesame 0x479cafb thru 0x47ab52c data_size 0x2(0xea32 bytes) 2024-09-12T07:20:34.905 INFO:teuthology.orchestra.run.smithi046.stdout:3711 write 0x4c7334a thru 0x4c78ff5 (0x5cac bytes) 2024-09-12T07:20:34.920 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:34.921+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:34.925 INFO:teuthology.orchestra.run.smithi046.stdout:3712 write 0xbfcd9d1 thru 0xbfd70f3 (0x9723 bytes) 2024-09-12T07:20:34.929 INFO:teuthology.orchestra.run.smithi046.stdout:3713 read 0x4abaaa1 thru 0x4ac3025 (0x8585 bytes) 2024-09-12T07:20:34.930 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.930 INFO:teuthology.orchestra.run.smithi046.stdout:3715 punch from 0xa0864c5 to 0xa08aa72, (0x45ad bytes) 2024-09-12T07:20:34.930 INFO:teuthology.orchestra.run.smithi046.stdout:3716 punch from 0x423ef2f to 0x4241517, (0x25e8 bytes) 2024-09-12T07:20:34.930 INFO:teuthology.orchestra.run.smithi046.stdout:3717 read 0x6687060 thru 0x6689a32 (0x29d3 bytes) 2024-09-12T07:20:34.930 INFO:teuthology.orchestra.run.smithi046.stdout:3718 read 0x3b9e369 thru 0x3ba6e71 (0x8b09 bytes) 2024-09-12T07:20:34.930 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:34.930 INFO:teuthology.orchestra.run.smithi046.stdout:3720 write 0x7eb4523 thru 0x7ec0973 (0xc451 bytes) 2024-09-12T07:20:34.935 INFO:teuthology.orchestra.run.smithi046.stdout:3721 writesame 0x2cf227a thru 0x2cf4729 data_size 0x8(0x24b0 bytes) 2024-09-12T07:20:34.939 INFO:teuthology.orchestra.run.smithi046.stdout:3724 write 0xbc61d17 thru 0xbc6ee3c (0xd126 bytes) 2024-09-12T07:20:34.948 INFO:teuthology.orchestra.run.smithi046.stdout:3725 trunc from 0xbfd70f4 to 0x75544c5 2024-09-12T07:20:34.954 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:34.961 INFO:teuthology.orchestra.run.smithi046.stdout:3726 write 0x6a1ebc9 thru 0x6a1fa35 (0xe6d bytes) 2024-09-12T07:20:34.972 INFO:teuthology.orchestra.run.smithi046.stdout:3727 write 0x91547ea thru 0x9158012 (0x3829 bytes) 2024-09-12T07:20:34.977 INFO:teuthology.orchestra.run.smithi046.stdout:3728 clone 27 order 20 su 8192 sc 5 2024-09-12T07:20:34.991 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:34.991 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:35.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:35.021 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:35.022+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:35.085 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:35.086 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:35.090 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:20:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:20:35.103 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-12T07:20:34.311899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":949127292,"num_objects":1226,"num_object_clones":296,"num_object_copies":2452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1226,"num_whiteouts":564,"num_read":164868,"num_read_kb":772868,"num_write":10422,"num_write_kb":413167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":49942512,"num_keys_recovered":48,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8516,"ondisk_log_size":8516,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":860872,"kb_used_data":559928,"kb_used_omap":12,"kb_used_meta":300659,"kb_avail":754113848,"statfs":{"total":773094113280,"available":772212580352,"internally_reserved":0,"allocated":573366272,"data_stored":676206150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":307875377},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":64279114,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":18768,"num_read_kb":122376,"num_write":2338,"num_write_kb":38842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017514"},"pg_stats":[{"pgid":"3.1f","version":"68'337","reported_seq":2020,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.304173+0000","last_change":"2024-09-12T07:19:44.455827+0000","last_active":"2024-09-12T07:20:32.304173+0000","last_peered":"2024-09-12T07:20:32.304173+0000","last_clean":"2024-09-12T07:20:32.304173+0000","last_became_active":"2024-09-12T07:19:42.451212+0000","last_became_peered":"2024-09-12T07:19:42.451212+0000","last_unstale":"2024-09-12T07:20:32.304173+0000","last_undegraded":"2024-09-12T07:20:32.304173+0000","last_fullsized":"2024-09-12T07:20:32.304173+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:04:31.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001023672,"stat_sum":{"num_bytes":13266909,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":3201,"num_read_kb":15912,"num_write":466,"num_write_kb":5844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"70'211","reported_seq":472,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.310404+0000","last_change":"2024-09-12T07:20:07.305432+0000","last_active":"2024-09-12T07:20:32.310404+0000","last_peered":"2024-09-12T07:20:32.310404+0000","last_clean":"2024-09-12T07:20:32.310404+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:20:32.310404+0000","last_undegraded":"2024-09-12T07:20:32.310404+0000","last_fullsized":"2024-09-12T07:20:32.310404+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:47.485606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081935099999999996,"stat_sum":{"num_bytes":45947739,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":17302,"num_read_kb":21358,"num_write":193,"num_write_kb":18619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"68'150","reported_seq":18011,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.312504+0000","last_change":"2024-09-12T07:19:44.464376+0000","last_active":"2024-09-12T07:20:32.312504+0000","last_peered":"2024-09-12T07:20:32.312504+0000","last_clean":"2024-09-12T07:20:32.312504+0000","last_became_active":"2024-09-12T07:19:42.451973+0000","last_became_peered":"2024-09-12T07:19:42.451973+0000","last_unstale":"2024-09-12T07:20:32.312504+0000","last_undegraded":"2024-09-12T07:20:32.312504+0000","last_fullsized":"2024-09-12T07:20:32.312504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:30:52.131510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036949999999999,"stat_sum":{"num_bytes":28202558,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":21,"num_read":17798,"num_read_kb":28430,"num_write":130,"num_write_kb":11932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"71'288","reported_seq":2107,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.476074+0000","last_change":"2024-09-12T07:19:44.464287+0000","last_active":"2024-09-12T07:20:32.476074+0000","last_peered":"2024-09-12T07:20:32.476074+0000","last_clean":"2024-09-12T07:20:32.476074+0000","last_became_active":"2024-09-12T07:19:42.455338+0000","last_became_peered":"2024-09-12T07:19:42.455338+0000","last_unstale":"2024-09-12T07:20:32.476074+0000","last_undegraded":"2024-09-12T07:20:32.476074+0000","last_fullsized":"2024-09-12T07:20:32.476074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:41.038321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012347180000000001,"stat_sum":{"num_bytes":26997538,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":2318,"num_read_kb":11364,"num_write":364,"num_write_kb":4866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"70'266","reported_seq":17721,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.533800+0000","last_change":"2024-09-12T07:19:44.456652+0000","last_active":"2024-09-12T07:20:32.533800+0000","last_peered":"2024-09-12T07:20:32.533800+0000","last_clean":"2024-09-12T07:20:32.533800+0000","last_became_active":"2024-09-12T07:19:42.454990+0000","last_became_peered":"2024-09-12T07:19:42.454990+0000","last_unstale":"2024-09-12T07:20:32.533800+0000","last_undegraded":"2024-09-12T07:20:32.533800+0000","last_fullsized":"2024-09-12T07:20:32.533800+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:29:56.750843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049866,"stat_sum":{"num_bytes":31412648,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":18271,"num_read_kb":45982,"num_write":280,"num_write_kb":14039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"71'242","reported_seq":2478,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.475673+0000","last_change":"2024-09-12T07:19:44.459417+0000","last_active":"2024-09-12T07:20:32.475673+0000","last_peered":"2024-09-12T07:20:32.475673+0000","last_clean":"2024-09-12T07:20:32.475673+0000","last_became_active":"2024-09-12T07:19:42.454620+0000","last_became_peered":"2024-09-12T07:19:42.454620+0000","last_unstale":"2024-09-12T07:20:32.475673+0000","last_undegraded":"2024-09-12T07:20:32.475673+0000","last_fullsized":"2024-09-12T07:20:32.475673+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:45:49.306030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090501500000000001,"stat_sum":{"num_bytes":15071959,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":22,"num_read":2993,"num_read_kb":9861,"num_write":230,"num_write_kb":14065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"68'299","reported_seq":4891,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.480530+0000","last_change":"2024-09-12T07:20:10.196744+0000","last_active":"2024-09-12T07:20:32.480530+0000","last_peered":"2024-09-12T07:20:32.480530+0000","last_clean":"2024-09-12T07:20:32.480530+0000","last_became_active":"2024-09-12T07:20:10.195312+0000","last_became_peered":"2024-09-12T07:20:10.195312+0000","last_unstale":"2024-09-12T07:20:32.480530+0000","last_undegraded":"2024-09-12T07:20:32.480530+0000","last_fullsized":"2024-09-12T07:20:32.480530+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:36.757545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113156,"stat_sum":{"num_bytes":27067089,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":5263,"num_read_kb":38807,"num_write":401,"num_write_kb":11737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1101525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"71'310","reported_seq":2142,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.460018+0000","last_change":"2024-09-12T07:20:07.446518+0000","last_active":"2024-09-12T07:20:32.460018+0000","last_peered":"2024-09-12T07:20:32.460018+0000","last_clean":"2024-09-12T07:20:32.460018+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:20:32.460018+0000","last_undegraded":"2024-09-12T07:20:32.460018+0000","last_fullsized":"2024-09-12T07:20:32.460018+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:40:40.304122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000443294,"stat_sum":{"num_bytes":40263224,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":4439,"num_read_kb":36222,"num_write":405,"num_write_kb":14391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"71'254","reported_seq":4440,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.533592+0000","last_change":"2024-09-12T07:19:44.456115+0000","last_active":"2024-09-12T07:20:32.533592+0000","last_peered":"2024-09-12T07:20:32.533592+0000","last_clean":"2024-09-12T07:20:32.533592+0000","last_became_active":"2024-09-12T07:19:42.452617+0000","last_became_peered":"2024-09-12T07:19:42.452617+0000","last_unstale":"2024-09-12T07:20:32.533592+0000","last_undegraded":"2024-09-12T07:20:32.533592+0000","last_fullsized":"2024-09-12T07:20:32.533592+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:31:01.422030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077251,"stat_sum":{"num_bytes":26742248,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":5357,"num_read_kb":12982,"num_write":236,"num_write_kb":20347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"68'122","reported_seq":400,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.312682+0000","last_change":"2024-09-12T07:19:44.463507+0000","last_active":"2024-09-12T07:20:32.312682+0000","last_peered":"2024-09-12T07:20:32.312682+0000","last_clean":"2024-09-12T07:20:32.312682+0000","last_became_active":"2024-09-12T07:19:42.455210+0000","last_became_peered":"2024-09-12T07:19:42.455210+0000","last_unstale":"2024-09-12T07:20:32.312682+0000","last_undegraded":"2024-09-12T07:20:32.312682+0000","last_fullsized":"2024-09-12T07:20:32.312682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:33.710716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049753700000000004,"stat_sum":{"num_bytes":14035109,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":20,"num_read":186,"num_read_kb":4520,"num_write":100,"num_write_kb":5250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"68'146","reported_seq":5930,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.536890+0000","last_change":"2024-09-12T07:20:07.261139+0000","last_active":"2024-09-12T07:20:32.536890+0000","last_peered":"2024-09-12T07:20:32.536890+0000","last_clean":"2024-09-12T07:20:32.536890+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:20:32.536890+0000","last_undegraded":"2024-09-12T07:20:32.536890+0000","last_fullsized":"2024-09-12T07:20:32.536890+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:52.445732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012225490000000001,"stat_sum":{"num_bytes":21134923,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":5928,"num_read_kb":41383,"num_write":127,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"68'206","reported_seq":4275,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.536650+0000","last_change":"2024-09-12T07:19:44.481796+0000","last_active":"2024-09-12T07:20:32.536650+0000","last_peered":"2024-09-12T07:20:32.536650+0000","last_clean":"2024-09-12T07:20:32.536650+0000","last_became_active":"2024-09-12T07:19:42.452277+0000","last_became_peered":"2024-09-12T07:19:42.452277+0000","last_unstale":"2024-09-12T07:20:32.536650+0000","last_undegraded":"2024-09-12T07:20:32.536650+0000","last_fullsized":"2024-09-12T07:20:32.536650+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:15:19.816581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012485160000000001,"stat_sum":{"num_bytes":33238886,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":4450,"num_read_kb":50918,"num_write":205,"num_write_kb":26359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":118,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.350602+0000","last_change":"2024-09-12T07:19:32.474512+0000","last_active":"2024-09-12T07:20:32.350602+0000","last_peered":"2024-09-12T07:20:32.350602+0000","last_clean":"2024-09-12T07:20:32.350602+0000","last_became_active":"2024-09-12T07:19:30.219990+0000","last_became_peered":"2024-09-12T07:19:30.219990+0000","last_unstale":"2024-09-12T07:20:32.350602+0000","last_undegraded":"2024-09-12T07:20:32.350602+0000","last_fullsized":"2024-09-12T07:20:32.350602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:32.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046339800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"68'319","reported_seq":1401,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.533039+0000","last_change":"2024-09-12T07:19:44.455257+0000","last_active":"2024-09-12T07:20:32.533039+0000","last_peered":"2024-09-12T07:20:32.533039+0000","last_clean":"2024-09-12T07:20:32.533039+0000","last_became_active":"2024-09-12T07:19:42.449730+0000","last_became_peered":"2024-09-12T07:19:42.449730+0000","last_unstale":"2024-09-12T07:20:32.533039+0000","last_undegraded":"2024-09-12T07:20:32.533039+0000","last_fullsized":"2024-09-12T07:20:32.533039+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:51:32.774607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059259600000000003,"stat_sum":{"num_bytes":18077355,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":22,"num_read":2062,"num_read_kb":9229,"num_write":399,"num_write_kb":8485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.306562+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:20:32.306562+0000","last_peered":"2024-09-12T07:20:32.306562+0000","last_clean":"2024-09-12T07:20:32.306562+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:20:32.306562+0000","last_undegraded":"2024-09-12T07:20:32.306562+0000","last_fullsized":"2024-09-12T07:20:32.306562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"70'281","reported_seq":993,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.310206+0000","last_change":"2024-09-12T07:20:07.211023+0000","last_active":"2024-09-12T07:20:32.310206+0000","last_peered":"2024-09-12T07:20:32.310206+0000","last_clean":"2024-09-12T07:20:32.310206+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:20:32.310206+0000","last_undegraded":"2024-09-12T07:20:32.310206+0000","last_fullsized":"2024-09-12T07:20:32.310206+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:11:36.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087343100000000003,"stat_sum":{"num_bytes":22892712,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":1462,"num_read_kb":12438,"num_write":364,"num_write_kb":5088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":118,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.333316+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:20:32.333316+0000","last_peered":"2024-09-12T07:20:32.333316+0000","last_clean":"2024-09-12T07:20:32.333316+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:20:32.333316+0000","last_undegraded":"2024-09-12T07:20:32.333316+0000","last_fullsized":"2024-09-12T07:20:32.333316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"68'247","reported_seq":950,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.531686+0000","last_change":"2024-09-12T07:19:44.458925+0000","last_active":"2024-09-12T07:20:32.531686+0000","last_peered":"2024-09-12T07:20:32.531686+0000","last_clean":"2024-09-12T07:20:32.531686+0000","last_became_active":"2024-09-12T07:19:42.452704+0000","last_became_peered":"2024-09-12T07:19:42.452704+0000","last_unstale":"2024-09-12T07:20:32.531686+0000","last_undegraded":"2024-09-12T07:20:32.531686+0000","last_fullsized":"2024-09-12T07:20:32.531686+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:16.578407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045577200000000001,"stat_sum":{"num_bytes":21841828,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":1516,"num_read_kb":4054,"num_write":313,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.306574+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:20:32.306574+0000","last_peered":"2024-09-12T07:20:32.306574+0000","last_clean":"2024-09-12T07:20:32.306574+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:20:32.306574+0000","last_undegraded":"2024-09-12T07:20:32.306574+0000","last_fullsized":"2024-09-12T07:20:32.306574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"68'220","reported_seq":4164,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.312584+0000","last_change":"2024-09-12T07:20:07.289858+0000","last_active":"2024-09-12T07:20:32.312584+0000","last_peered":"2024-09-12T07:20:32.312584+0000","last_clean":"2024-09-12T07:20:32.312584+0000","last_became_active":"2024-09-12T07:20:07.183819+0000","last_became_peered":"2024-09-12T07:20:07.183819+0000","last_unstale":"2024-09-12T07:20:32.312584+0000","last_undegraded":"2024-09-12T07:20:32.312584+0000","last_fullsized":"2024-09-12T07:20:32.312584+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:26.537525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015134619999999999,"stat_sum":{"num_bytes":45604391,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":4626,"num_read_kb":44381,"num_write":258,"num_write_kb":18869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":3070354,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"68'149","reported_seq":1611,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.533620+0000","last_change":"2024-09-12T07:19:44.456465+0000","last_active":"2024-09-12T07:20:32.533620+0000","last_peered":"2024-09-12T07:20:32.533620+0000","last_clean":"2024-09-12T07:20:32.533620+0000","last_became_active":"2024-09-12T07:19:42.454103+0000","last_became_peered":"2024-09-12T07:19:42.454103+0000","last_unstale":"2024-09-12T07:20:32.533620+0000","last_undegraded":"2024-09-12T07:20:32.533620+0000","last_fullsized":"2024-09-12T07:20:32.533620+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:15.119417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010980880000000001,"stat_sum":{"num_bytes":18636911,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":24,"num_read":1378,"num_read_kb":3209,"num_write":127,"num_write_kb":4808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":120,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.312367+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:20:32.312367+0000","last_peered":"2024-09-12T07:20:32.312367+0000","last_clean":"2024-09-12T07:20:32.312367+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:20:32.312367+0000","last_undegraded":"2024-09-12T07:20:32.312367+0000","last_fullsized":"2024-09-12T07:20:32.312367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"68'271","reported_seq":2000,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.306616+0000","last_change":"2024-09-12T07:19:44.458868+0000","last_active":"2024-09-12T07:20:32.306616+0000","last_peered":"2024-09-12T07:20:32.306616+0000","last_clean":"2024-09-12T07:20:32.306616+0000","last_became_active":"2024-09-12T07:19:42.451807+0000","last_became_peered":"2024-09-12T07:19:42.451807+0000","last_unstale":"2024-09-12T07:20:32.306616+0000","last_undegraded":"2024-09-12T07:20:32.306616+0000","last_fullsized":"2024-09-12T07:20:32.306616+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:41:18.028635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048363399999999998,"stat_sum":{"num_bytes":29481674,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":25,"num_read":2960,"num_read_kb":8628,"num_write":336,"num_write_kb":9392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":130,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.447351+0000","last_change":"2024-09-12T07:20:07.183543+0000","last_active":"2024-09-12T07:20:32.447351+0000","last_peered":"2024-09-12T07:20:32.447351+0000","last_clean":"2024-09-12T07:20:32.447351+0000","last_became_active":"2024-09-12T07:20:07.183213+0000","last_became_peered":"2024-09-12T07:20:07.183213+0000","last_unstale":"2024-09-12T07:20:32.447351+0000","last_undegraded":"2024-09-12T07:20:32.447351+0000","last_fullsized":"2024-09-12T07:20:32.447351+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.634459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"68'144","reported_seq":2853,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.303766+0000","last_change":"2024-09-12T07:19:44.455560+0000","last_active":"2024-09-12T07:20:32.303766+0000","last_peered":"2024-09-12T07:20:32.303766+0000","last_clean":"2024-09-12T07:20:32.303766+0000","last_became_active":"2024-09-12T07:19:42.451970+0000","last_became_peered":"2024-09-12T07:19:42.451970+0000","last_unstale":"2024-09-12T07:20:32.303766+0000","last_undegraded":"2024-09-12T07:20:32.303766+0000","last_fullsized":"2024-09-12T07:20:32.303766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:05:25.259536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083352100000000004,"stat_sum":{"num_bytes":43142164,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":3547,"num_read_kb":33681,"num_write":158,"num_write_kb":23103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":118,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.332995+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:20:32.332995+0000","last_peered":"2024-09-12T07:20:32.332995+0000","last_clean":"2024-09-12T07:20:32.332995+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:20:32.332995+0000","last_undegraded":"2024-09-12T07:20:32.332995+0000","last_fullsized":"2024-09-12T07:20:32.332995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"68'688","reported_seq":3038,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.310137+0000","last_change":"2024-09-12T07:20:07.467030+0000","last_active":"2024-09-12T07:20:32.310137+0000","last_peered":"2024-09-12T07:20:32.310137+0000","last_clean":"2024-09-12T07:20:32.310137+0000","last_became_active":"2024-09-12T07:20:07.180577+0000","last_became_peered":"2024-09-12T07:20:07.180577+0000","last_unstale":"2024-09-12T07:20:32.310137+0000","last_undegraded":"2024-09-12T07:20:32.310137+0000","last_fullsized":"2024-09-12T07:20:32.310137+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:24:11.168066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089638599999999999,"stat_sum":{"num_bytes":32630331,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":22,"num_read":5273,"num_read_kb":33681,"num_write":1110,"num_write_kb":14584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":261251,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":119,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.312384+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:20:32.312384+0000","last_peered":"2024-09-12T07:20:32.312384+0000","last_clean":"2024-09-12T07:20:32.312384+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:20:32.312384+0000","last_undegraded":"2024-09-12T07:20:32.312384+0000","last_fullsized":"2024-09-12T07:20:32.312384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":209,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.333010+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:20:32.333010+0000","last_peered":"2024-09-12T07:20:32.333010+0000","last_clean":"2024-09-12T07:20:32.333010+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:20:32.333010+0000","last_undegraded":"2024-09-12T07:20:32.333010+0000","last_fullsized":"2024-09-12T07:20:32.333010+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"71'756","reported_seq":1516,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.477577+0000","last_change":"2024-09-12T07:20:07.428482+0000","last_active":"2024-09-12T07:20:32.477577+0000","last_peered":"2024-09-12T07:20:32.477577+0000","last_clean":"2024-09-12T07:20:32.477577+0000","last_became_active":"2024-09-12T07:20:07.180763+0000","last_became_peered":"2024-09-12T07:20:07.180763+0000","last_unstale":"2024-09-12T07:20:32.477577+0000","last_undegraded":"2024-09-12T07:20:32.477577+0000","last_fullsized":"2024-09-12T07:20:32.477577+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:02.054433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047491800000000001,"stat_sum":{"num_bytes":33165368,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":4144,"num_read_kb":20152,"num_write":1269,"num_write_kb":18039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":8633990,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"68'314","reported_seq":3447,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.447475+0000","last_change":"2024-09-12T07:19:44.469508+0000","last_active":"2024-09-12T07:20:32.447475+0000","last_peered":"2024-09-12T07:20:32.447475+0000","last_clean":"2024-09-12T07:20:32.447475+0000","last_became_active":"2024-09-12T07:19:42.450230+0000","last_became_peered":"2024-09-12T07:19:42.450230+0000","last_unstale":"2024-09-12T07:20:32.447475+0000","last_undegraded":"2024-09-12T07:20:32.447475+0000","last_fullsized":"2024-09-12T07:20:32.447475+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:34:50.649525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076460899999999997,"stat_sum":{"num_bytes":31927129,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":23,"num_read":5084,"num_read_kb":25900,"num_write":314,"num_write_kb":3106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"71'183","reported_seq":3236,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.350767+0000","last_change":"2024-09-12T07:19:44.480981+0000","last_active":"2024-09-12T07:20:32.350767+0000","last_peered":"2024-09-12T07:20:32.350767+0000","last_clean":"2024-09-12T07:20:32.350767+0000","last_became_active":"2024-09-12T07:19:42.452637+0000","last_became_peered":"2024-09-12T07:19:42.452637+0000","last_unstale":"2024-09-12T07:20:32.350767+0000","last_undegraded":"2024-09-12T07:20:32.350767+0000","last_fullsized":"2024-09-12T07:20:32.350767+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:10.659877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000680892,"stat_sum":{"num_bytes":35360399,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":25,"num_read":3083,"num_read_kb":31017,"num_write":197,"num_write_kb":13093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"70'219","reported_seq":1293,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.310327+0000","last_change":"2024-09-12T07:20:07.379907+0000","last_active":"2024-09-12T07:20:32.310327+0000","last_peered":"2024-09-12T07:20:32.310327+0000","last_clean":"2024-09-12T07:20:32.310327+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:20:32.310327+0000","last_undegraded":"2024-09-12T07:20:32.310327+0000","last_fullsized":"2024-09-12T07:20:32.310327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:35.725520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054644100000000005,"stat_sum":{"num_bytes":28454093,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":20,"num_read":1991,"num_read_kb":11621,"num_write":221,"num_write_kb":13079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"70'225","reported_seq":7468,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.531224+0000","last_change":"2024-09-12T07:20:07.249347+0000","last_active":"2024-09-12T07:20:32.531224+0000","last_peered":"2024-09-12T07:20:32.531224+0000","last_clean":"2024-09-12T07:20:32.531224+0000","last_became_active":"2024-09-12T07:20:07.183065+0000","last_became_peered":"2024-09-12T07:20:07.183065+0000","last_unstale":"2024-09-12T07:20:32.531224+0000","last_undegraded":"2024-09-12T07:20:32.531224+0000","last_fullsized":"2024-09-12T07:20:32.531224+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:47.361795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050446300000000003,"stat_sum":{"num_bytes":39840262,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":7879,"num_read_kb":42906,"num_write":199,"num_write_kb":14091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":8578058,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"68'159","reported_seq":1718,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.333251+0000","last_change":"2024-09-12T07:19:44.482470+0000","last_active":"2024-09-12T07:20:32.333251+0000","last_peered":"2024-09-12T07:20:32.333251+0000","last_clean":"2024-09-12T07:20:32.333251+0000","last_became_active":"2024-09-12T07:19:42.450577+0000","last_became_peered":"2024-09-12T07:19:42.450577+0000","last_unstale":"2024-09-12T07:20:32.333251+0000","last_undegraded":"2024-09-12T07:20:32.333251+0000","last_fullsized":"2024-09-12T07:20:32.333251+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:44:46.567235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317836,"stat_sum":{"num_bytes":34488353,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":1814,"num_read_kb":19313,"num_write":175,"num_write_kb":8379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"71'326","reported_seq":3745,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.532247+0000","last_change":"2024-09-12T07:19:44.481850+0000","last_active":"2024-09-12T07:20:32.532247+0000","last_peered":"2024-09-12T07:20:32.532247+0000","last_clean":"2024-09-12T07:20:32.532247+0000","last_became_active":"2024-09-12T07:19:42.452236+0000","last_became_peered":"2024-09-12T07:19:42.452236+0000","last_unstale":"2024-09-12T07:20:32.532247+0000","last_undegraded":"2024-09-12T07:20:32.532247+0000","last_fullsized":"2024-09-12T07:20:32.532247+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:44:50.439574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012939710000000001,"stat_sum":{"num_bytes":21428311,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":5182,"num_read_kb":23521,"num_write":424,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"68'319","reported_seq":1026,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.536511+0000","last_change":"2024-09-12T07:19:44.482179+0000","last_active":"2024-09-12T07:20:32.536511+0000","last_peered":"2024-09-12T07:20:32.536511+0000","last_clean":"2024-09-12T07:20:32.536511+0000","last_became_active":"2024-09-12T07:19:42.452155+0000","last_became_peered":"2024-09-12T07:19:42.452155+0000","last_unstale":"2024-09-12T07:20:32.536511+0000","last_undegraded":"2024-09-12T07:20:32.536511+0000","last_fullsized":"2024-09-12T07:20:32.536511+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:06:49.990730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011884759999999999,"stat_sum":{"num_bytes":15047103,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":1807,"num_read_kb":7338,"num_write":397,"num_write_kb":14969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"70'187","reported_seq":657,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.532297+0000","last_change":"2024-09-12T07:19:44.481020+0000","last_active":"2024-09-12T07:20:32.532297+0000","last_peered":"2024-09-12T07:20:32.532297+0000","last_clean":"2024-09-12T07:20:32.532297+0000","last_became_active":"2024-09-12T07:19:42.450535+0000","last_became_peered":"2024-09-12T07:19:42.450535+0000","last_unstale":"2024-09-12T07:20:32.532297+0000","last_undegraded":"2024-09-12T07:20:32.532297+0000","last_fullsized":"2024-09-12T07:20:32.532297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:54.851331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073032100000000003,"stat_sum":{"num_bytes":43465959,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":719,"num_read_kb":6958,"num_write":212,"num_write_kb":16344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"68'165","reported_seq":2759,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.531426+0000","last_change":"2024-09-12T07:19:44.481275+0000","last_active":"2024-09-12T07:20:32.531426+0000","last_peered":"2024-09-12T07:20:32.531426+0000","last_clean":"2024-09-12T07:20:32.531426+0000","last_became_active":"2024-09-12T07:19:42.452104+0000","last_became_peered":"2024-09-12T07:19:42.452104+0000","last_unstale":"2024-09-12T07:20:32.531426+0000","last_undegraded":"2024-09-12T07:20:32.531426+0000","last_fullsized":"2024-09-12T07:20:32.531426+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:55:17.910307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079083699999999996,"stat_sum":{"num_bytes":18167110,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":2973,"num_read_kb":30025,"num_write":156,"num_write_kb":13865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"68'123","reported_seq":1571,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.303769+0000","last_change":"2024-09-12T07:19:44.456086+0000","last_active":"2024-09-12T07:20:32.303769+0000","last_peered":"2024-09-12T07:20:32.303769+0000","last_clean":"2024-09-12T07:20:32.303769+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:20:32.303769+0000","last_undegraded":"2024-09-12T07:20:32.303769+0000","last_fullsized":"2024-09-12T07:20:32.303769+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:09.412505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060746,"stat_sum":{"num_bytes":30934923,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":12,"num_read":1342,"num_read_kb":17954,"num_write":159,"num_write_kb":9424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"70'315","reported_seq":18272,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T07:20:32.536282+0000","last_change":"2024-09-12T07:19:44.482564+0000","last_active":"2024-09-12T07:20:32.536282+0000","last_peered":"2024-09-12T07:20:32.536282+0000","last_clean":"2024-09-12T07:20:32.536282+0000","last_became_active":"2024-09-12T07:19:42.456127+0000","last_became_peered":"2024-09-12T07:19:42.456127+0000","last_unstale":"2024-09-12T07:20:32.536282+0000","last_undegraded":"2024-09-12T07:20:32.536282+0000","last_fullsized":"2024-09-12T07:20:32.536282+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:48:43.140119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793039999999999,"stat_sum":{"num_bytes":60569699,"num_objects":51,"num_object_clones":16,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":18454,"num_read_kb":69068,"num_write":394,"num_write_kb":33928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":948536905,"num_objects":1222,"num_object_clones":296,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":564,"num_read":164802,"num_read_kb":772813,"num_write":10319,"num_write_kb":411537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":49942512,"num_keys_recovered":48,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565641216,"data_stored":672083868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8441,"ondisk_log_size":8441,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":128180,"kb_used_data":92832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94243660,"statfs":{"total":96636764160,"available":96505507840,"internally_reserved":0,"allocated":95059968,"data_stored":110884325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63352,"kb_used_data":32056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308488,"statfs":{"total":96636764160,"available":96571891712,"internally_reserved":0,"allocated":32825344,"data_stored":39034069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114300,"kb_used_data":78944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94257540,"statfs":{"total":96636764160,"available":96519720960,"internally_reserved":0,"allocated":80838656,"data_stored":95025823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179168,"kb_used_data":139660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94192672,"statfs":{"total":96636764160,"available":96453296128,"internally_reserved":0,"allocated":143011840,"data_stored":177291358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134004,"kb_used_data":88932,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94237836,"statfs":{"total":96636764160,"available":96499544064,"internally_reserved":0,"allocated":91066368,"data_stored":106265436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46692,"kb_used_data":11344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325148,"statfs":{"total":96636764160,"available":96588951552,"internally_reserved":0,"allocated":11616256,"data_stored":11092893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72332,"kb_used_data":32820,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94299508,"statfs":{"total":96636764160,"available":96562696192,"internally_reserved":0,"allocated":33607680,"data_stored":34627208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122844,"kb_used_data":83340,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94248996,"statfs":{"total":96636764160,"available":96510971904,"internally_reserved":0,"allocated":85340160,"data_stored":101985038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":83935232,"data_stored":101028548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32788480,"data_stored":34261020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10788864,"data_stored":10720573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":90251264,"data_stored":105899267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":142184448,"data_stored":176919038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":80027648,"data_stored":94659718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32010240,"data_stored":38667900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":93655040,"data_stored":109927804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:20:35.105 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-12T07:20:35.105 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:20:35.105 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-12T07:20:35.106 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.000000 2024-09-12T07:20:35.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0 2024-09-12T07:20:35.106 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:35.107 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:35.121 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:35.121 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:35.121 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:35.123+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:35.153 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:35.154 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:35.168 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:35.168 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126196 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6471671299999997, 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038184672000000003, 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126196 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:35.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6199999999999999e-07 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521351+0000", 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9520000000000001e-06 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521357+0000", 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521362+0000", 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9180000000000002e-06 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559327+0000", 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037964880999999999 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559401+0000", 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3750000000000004e-05 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559533+0000", 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013262300000000001 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126334 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6465863929999998, 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038163516000000001, 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126334 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521930+0000", 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521932+0000", 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9420000000000002e-06 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521938+0000", 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.127e-06 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521949+0000", 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1538e-05 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559814+0000", 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037864291000000001 2024-09-12T07:20:35.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559903+0000", 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9409999999999999e-05 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560093+0000", 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018984899999999999 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126336 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6465581560000002, 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038384952, 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126336 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.178 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2e-07 2024-09-12T07:20:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521962+0000", 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.591e-06 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521969+0000", 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9199999999999998e-06 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521974+0000", 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2179999999999998e-06 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560137+0000", 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038163034999999998 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560193+0000", 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6425000000000003e-05 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560343+0000", 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000149343 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126337 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6465350450000003, 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038587955, 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126337 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521983+0000", 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2079999999999999e-06 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521989+0000", 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8649999999999998e-06 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521994+0000", 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8720000000000001e-06 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560393+0000", 2024-09-12T07:20:35.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038399232999999998 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560442+0000", 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8310999999999999e-05 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560569+0000", 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012708700000000001 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.186 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126339 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6465151709999999, 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038809422000000003, 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126339 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:35.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.58e-07 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522003+0000", 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0059999999999999e-06 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522009+0000", 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522014+0000", 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0409999999999997e-06 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560605+0000", 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038590994000000003 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560650+0000", 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5336000000000001e-05 2024-09-12T07:20:35.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560810+0000", 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000159853 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126341 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6464958420000002, 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039112779, 2024-09-12T07:20:35.191 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126341 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6399999999999998e-07 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522022+0000", 2024-09-12T07:20:35.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522028+0000", 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9089999999999997e-06 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522035+0000", 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5200000000000003e-06 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560857+0000", 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038822239000000001 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560955+0000", 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7503999999999993e-05 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561133+0000", 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017827899999999999 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.195 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126342 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6464739059999998, 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039296258000000001, 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126342 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.84e-07 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522044+0000", 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522057+0000", 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330999999999999e-05 2024-09-12T07:20:35.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522062+0000", 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3700000000000003e-06 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561151+0000", 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039089193000000001 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561215+0000", 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3305000000000005e-05 2024-09-12T07:20:35.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561338+0000", 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012353900000000001 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126343 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6464457880000003, 2024-09-12T07:20:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039573481000000001, 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126343 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522071+0000", 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2599999999999998e-07 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522077+0000", 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2049999999999999e-06 2024-09-12T07:20:35.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522084+0000", 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1679999999999998e-06 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522089+0000", 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561375+0000", 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039286692999999998 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561463+0000", 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7575000000000001e-05 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561644+0000", 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018070799999999999 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.204 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126344 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6464192720000002, 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039759638, 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126344 2024-09-12T07:20:35.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4900000000000001e-07 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522099+0000", 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4250000000000001e-06 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522106+0000", 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.455e-06 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522112+0000", 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3169999999999998e-06 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561676+0000", 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039564153999999997 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561728+0000", 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1848999999999998e-05 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561856+0000", 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012808900000000001 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126345 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6463926789999999, 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040055773000000003, 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.209 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126345 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522124+0000", 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6600000000000002e-07 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522130+0000", 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.72e-06 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522140+0000", 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0217000000000001e-05 2024-09-12T07:20:35.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522146+0000", 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3380000000000004e-06 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561896+0000", 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039750343 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561997+0000", 2024-09-12T07:20:35.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010129899999999999 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562179+0000", 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018149 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126346 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6461854440000003, 2024-09-12T07:20:35.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040058279000000002, 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126346 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522331+0000", 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522334+0000", 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.01e-06 2024-09-12T07:20:35.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522346+0000", 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522353+0000", 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5590000000000002e-06 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562212+0000", 2024-09-12T07:20:35.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039859232000000001 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562256+0000", 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4354999999999998e-05 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562389+0000", 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013237899999999999 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126348 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6461527589999996, 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040352420999999999, 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126348 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522364+0000", 2024-09-12T07:20:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.51e-07 2024-09-12T07:20:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522366+0000", 2024-09-12T07:20:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522373+0000", 2024-09-12T07:20:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3339999999999996e-06 2024-09-12T07:20:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522378+0000", 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8620000000000002e-06 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562428+0000", 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040049840000000003 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562522+0000", 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3589000000000001e-05 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562716+0000", 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019395799999999999 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126349 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1269760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6461299509999998, 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040535117000000002, 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126349 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522389+0000", 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.317e-06 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522396+0000", 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522401+0000", 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5770000000000001e-06 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562754+0000", 2024-09-12T07:20:35.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040352271000000002 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562809+0000", 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5376e-05 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562921+0000", 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112098 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126350 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6461074619999998, 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040836117999999998, 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126350 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522409+0000", 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9099999999999999e-07 2024-09-12T07:20:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522411+0000", 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1449999999999998e-06 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522417+0000", 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1070000000000001e-06 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522422+0000", 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7600000000000002e-06 2024-09-12T07:20:35.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562965+0000", 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040543467 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563058+0000", 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2905000000000006e-05 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563245+0000", 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186343 2024-09-12T07:20:35.230 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126351 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6459166209999996, 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040915973000000001, 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.231 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126351 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522600+0000", 2024-09-12T07:20:35.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7899999999999998e-07 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522604+0000", 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-06 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522615+0000", 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1754e-05 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522622+0000", 2024-09-12T07:20:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.798e-06 2024-09-12T07:20:35.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563277+0000", 2024-09-12T07:20:35.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040654765000000002 2024-09-12T07:20:35.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563340+0000", 2024-09-12T07:20:35.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2580000000000001e-05 2024-09-12T07:20:35.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563515+0000", 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017560699999999999 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126352 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6458813269999997, 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041184365000000001, 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126352 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.99e-07 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522637+0000", 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3180000000000001e-06 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522645+0000", 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0309999999999998e-06 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522651+0000", 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.789e-06 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563559+0000", 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040907261 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563634+0000", 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5708000000000001e-05 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563819+0000", 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000184659 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.239 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126353 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.6458558740000004, 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388632000000002, 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126353 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522661+0000", 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7299999999999996e-07 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522663+0000", 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522670+0000", 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6850000000000004e-06 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522675+0000", 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9139999999999999e-06 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563854+0000", 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041178851000000002 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563912+0000", 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8584e-05 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564049+0000", 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000136427 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126354 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.645833348, 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041618080000000002, 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126354 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.244 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:35.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2399999999999999e-07 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522685+0000", 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2469999999999998e-06 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522691+0000", 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.353e-06 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522697+0000", 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.113e-06 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564085+0000", 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388104000000002 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564149+0000", 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4471e-05 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564301+0000", 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015146800000000001 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:129861 3.18 3:1cf16ce6:::rbd_data.116031710b3d.00000000000000ae:18 [read 40960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.395897637, 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038044815000000003, 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.248 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 129861 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:35.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772619+0000", 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5400000000000002e-07 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772620+0000", 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6980000000000001e-06 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772627+0000", 2024-09-12T07:20:35.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3119999999999999e-06 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772631+0000", 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9010000000000002e-06 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810471+0000", 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037839308000000002 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810521+0000", 2024-09-12T07:20:35.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9988999999999999e-05 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810663+0000", 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014225300000000001 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:129880 3.8 3:149a3dec:::rbd_data.116031710b3d.00000000000000ab:18 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:35.252 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.3958574209999997, 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038109654999999999, 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 129880 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:35.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772659+0000", 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772661+0000", 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8279999999999999e-06 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772666+0000", 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6319999999999996e-06 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772672+0000", 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0510000000000002e-06 2024-09-12T07:20:35.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810577+0000", 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037904895000000001 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810688+0000", 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011072400000000001 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810768+0000", 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0139e-05 2024-09-12T07:20:35.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.257 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.257 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.257 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:35.257 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:35.257 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:35.257 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:35.259 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:35.259 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:35.260 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:35.260 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:35.261 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:35.259+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:35.313 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:20:35.359 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:35.360+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:35.377 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:35.377 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:35.377 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:35.378 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:35.386 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:35.471 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-12T07:20:35.471 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:35.473+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:35.498 INFO:teuthology.orchestra.run.smithi046.stdout:3729 write 0xc6e5db7 thru 0xc6e9c6d (0x3eb7 bytes) 2024-09-12T07:20:35.509 INFO:teuthology.orchestra.run.smithi046.stdout:3732 trunc from 0xc6e9c6e to 0x8850369 2024-09-12T07:20:35.539 INFO:teuthology.orchestra.run.smithi046.stdout:3733 trunc from 0x8850369 to 0xb96ad2e 2024-09-12T07:20:35.541 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:35.541 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:35.541 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:35.541 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:35.541 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:35.545 INFO:teuthology.orchestra.run.smithi046.stdout:3734 trunc from 0xb96ad2e to 0xcd1379e 2024-09-12T07:20:35.550 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:35.551 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:35.551 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:35.552 INFO:teuthology.orchestra.run.smithi046.stdout:3738 write 0xa31c55 thru 0xa3bfe1 (0xa38d bytes) 2024-09-12T07:20:35.585 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:35.585 INFO:teuthology.orchestra.run.smithi046.stdout:3740 write 0x53932f7 thru 0x539899d (0x56a7 bytes) 2024-09-12T07:20:35.591 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:35.593+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:35.604 INFO:teuthology.orchestra.run.smithi046.stdout:3742 read 0x7ca4a9d thru 0x7ca74f2 (0x2a56 bytes) 2024-09-12T07:20:35.604 INFO:teuthology.orchestra.run.smithi046.stdout:3743 write 0x7850bd8 thru 0x7859a24 (0x8e4d bytes) 2024-09-12T07:20:35.624 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:35.624 INFO:teuthology.orchestra.run.smithi046.stdout:3745 punch from 0x8fd3e8f to 0x8fdfe6d, (0xbfde bytes) 2024-09-12T07:20:35.624 INFO:teuthology.orchestra.run.smithi046.stdout:3747 read 0x107d0bb thru 0x107efd9 (0x1f1f bytes) 2024-09-12T07:20:35.631 INFO:teuthology.orchestra.run.smithi046.stdout:3748 write 0xea66ba7 thru 0xea72bed (0xc047 bytes) 2024-09-12T07:20:35.638 INFO:teuthology.orchestra.run.smithi046.stdout:3751 read 0x26c3271 thru 0x26ccfc4 (0x9d54 bytes) 2024-09-12T07:20:35.640 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:35.640 INFO:teuthology.orchestra.run.smithi046.stdout:3756 write 0x3dd8adc thru 0x3de30e8 (0xa60d bytes) 2024-09-12T07:20:35.658 INFO:teuthology.orchestra.run.smithi046.stdout:3758 write 0x8f9d206 thru 0x8fa2681 (0x547c bytes) 2024-09-12T07:20:35.661 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:35.662+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:35.663 INFO:teuthology.orchestra.run.smithi046.stdout:3759 write 0x6493c68 thru 0x649665b (0x29f4 bytes) 2024-09-12T07:20:35.677 INFO:teuthology.orchestra.run.smithi046.stdout:3760 punch from 0x498934b to 0x4994af7, (0xb7ac bytes) 2024-09-12T07:20:35.677 INFO:teuthology.orchestra.run.smithi046.stdout:3761 read 0xd64f58b thru 0xd651ca2 (0x2718 bytes) 2024-09-12T07:20:35.678 INFO:teuthology.orchestra.run.smithi046.stdout:3763 read 0x7d847de thru 0x7d8dbe2 (0x9405 bytes) 2024-09-12T07:20:35.678 INFO:teuthology.orchestra.run.smithi046.stdout:3764 writesame 0xa9db08b thru 0xa9e4129 data_size 0x7b(0x909f bytes) 2024-09-12T07:20:35.684 INFO:teuthology.orchestra.run.smithi046.stdout:3765 writesame 0x2bf0d16 thru 0x2bff9c1 data_size 0x51(0xecac bytes) 2024-09-12T07:20:35.723 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:35.723 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:35.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:35.723 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:35.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.723 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:20:35.723 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:35.723 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 38.110732046999999, 2024-09-12T07:20:35.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:20:35.723 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.723 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.975271790999997, 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:35.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:20:35.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:20:35.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:20:35.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.971044669999998, 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:20:35.732 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.243294014999996, 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:20:35.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:20:35.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:20:35.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:20:35.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:20:35.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:20:35.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:35.741 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.154317425999999, 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:35.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:20:35.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68853 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'43, mlcod=43'39)", 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.143324839000002, 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017793166999999999, 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:35.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:35.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580719+0000", 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4720000000000001e-06 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580732+0000", 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2926e-05 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580746+0000", 2024-09-12T07:20:35.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3291e-05 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580755+0000", 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6709999999999994e-06 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583766+0000", 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003010432 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583794+0000", 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.811e-05 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598408+0000", 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014614472 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598511+0000", 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102793 2024-09-12T07:20:35.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.122766095999999, 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:35.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.755 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.119565559999998, 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:35.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:20:35.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:20:35.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:20:35.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.116319812, 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.760 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.114077981000001, 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:20:35.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:20:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69559 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'29, mlcod=41'27)", 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.039746933000004, 2024-09-12T07:20:35.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018119258999999999, 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684297+0000", 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.192e-06 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691509+0000", 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072122239999999997 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691524+0000", 2024-09-12T07:20:35.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4562999999999999e-05 2024-09-12T07:20:35.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691533+0000", 2024-09-12T07:20:35.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1069999999999994e-06 2024-09-12T07:20:35.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691598+0000", 2024-09-12T07:20:35.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4666999999999996e-05 2024-09-12T07:20:35.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691620+0000", 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2092000000000001e-05 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702352+0000", 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010731785000000001 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702415+0000", 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3628999999999997e-05 2024-09-12T07:20:35.773 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69560 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'31, mlcod=41'27)", 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.03218081, 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016047555000000002, 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691863+0000", 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0299999999999999e-07 2024-09-12T07:20:35.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691897+0000", 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4765999999999999e-05 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691926+0000", 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8413000000000001e-05 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691934+0000", 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3839999999999997e-06 2024-09-12T07:20:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694739+0000", 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028043640000000002 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694773+0000", 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4514000000000002e-05 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707854+0000", 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013080591000000001 2024-09-12T07:20:35.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707910+0000", 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.592e-05 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "pg_update_log_missing(3.1a epoch 43/24 rep_tid 362 entries 43'32 (0'0) error 3:5ec4a676:::rbd_data.1160f022dd61.0000000000000019:head by client.4448.0:69639 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 41'27)", 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.031653095999999, 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015537618, 2024-09-12T07:20:35.778 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692497+0000", 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107154 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692519+0000", 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1043000000000002e-05 2024-09-12T07:20:35.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692542+0000", 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3866000000000002e-05 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702496+0000", 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0099539169999999996 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707927+0000", 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005431038 2024-09-12T07:20:35.781 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.022970776999998, 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:20:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:20:35.786 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.019142688999999, 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:20:35.790 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89144 3.a e53/24 3:54996542:::rbd_data.11607a56846.0000000000000001:head v 53'83, mlcod=53'81)", 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 22.030985935, 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018298937000000001, 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693059+0000", 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.565e-06 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696238+0000", 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0031783269999999999 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696265+0000", 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6982e-05 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696279+0000", 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4438e-05 2024-09-12T07:20:35.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696366+0000", 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6978000000000006e-05 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696397+0000", 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0406999999999999e-05 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711297+0000", 2024-09-12T07:20:35.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014900689999999999 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711356+0000", 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.855e-05 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89145 3.13 e53/24 3:ca7c3842:::rbd_data.11607a56846.0000000000000002:head v 53'202, mlcod=53'199)", 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 22.029954591999999, 2024-09-12T07:20:35.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019835603, 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:35.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694091+0000", 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4990000000000001e-06 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697616+0000", 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035256749999999998 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697635+0000", 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8593999999999998e-05 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697646+0000", 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1436e-05 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697717+0000", 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0679000000000005e-05 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697748+0000", 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0723999999999999e-05 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713886+0000", 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016138550000000002 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713924+0000", 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7446e-05 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.799 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.0372299080000005, 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.0254162640000004, 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:20:35.804 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:20:35.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.0240802169999998, 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:20:35.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:35.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:35.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:20:35.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:20:35.813 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.813 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.813 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.813 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.813 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.813 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:35.814 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:35.815 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:35.816+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:35.915 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:35.917+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:35.925 INFO:teuthology.orchestra.run.smithi046.stdout:3766 trunc from 0xea72bee to 0x32b35a1 2024-09-12T07:20:35.967 INFO:teuthology.orchestra.run.smithi046.stdout:3767 read 0x2204ce8 thru 0x220e61c (0x9935 bytes) 2024-09-12T07:20:35.968 INFO:teuthology.orchestra.run.smithi046.stdout:3768 writesame 0x8b9249 thru 0x8c3f2d data_size 0x1(0xace5 bytes) 2024-09-12T07:20:35.992 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147547 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.984650+0000", 2024-09-12T07:20:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0028735950000000001, 2024-09-12T07:20:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028974830000000002, 2024-09-12T07:20:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147547 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984650+0000", 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984650+0000", 2024-09-12T07:20:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984651+0000", 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5900000000000002e-07 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984653+0000", 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3190000000000002e-06 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984660+0000", 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9179999999999997e-06 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984675+0000", 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5608e-05 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147643 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985440+0000", 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0020838900000000001, 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021739799999999998, 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147643 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985440+0000", 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985440+0000", 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985440+0000", 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2800000000000002e-07 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985452+0000", 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2016e-05 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985463+0000", 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0654e-05 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985469+0000", 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.998 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147675 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986398+0000", 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0011260669999999999, 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001269109, 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:35.999 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147675 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986398+0000", 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986398+0000", 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986399+0000", 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4929999999999999e-06 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986401+0000", 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3410000000000001e-06 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986408+0000", 2024-09-12T07:20:36.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9439999999999999e-06 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986413+0000", 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-06 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147771 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987234+0000", 2024-09-12T07:20:36.002 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00028989700000000001, 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00048430899999999999, 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147771 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987234+0000", 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987234+0000", 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987234+0000", 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8699999999999995e-07 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987242+0000", 2024-09-12T07:20:36.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8220000000000007e-06 2024-09-12T07:20:36.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987249+0000", 2024-09-12T07:20:36.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0029999999999997e-06 2024-09-12T07:20:36.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987254+0000", 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.237e-06 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147548 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.984685+0000", 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0028389499999999998, 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030913529999999998, 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.006 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147548 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984685+0000", 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984685+0000", 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984685+0000", 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0299999999999999e-07 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984688+0000", 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4169999999999999e-06 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984694+0000", 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4080000000000003e-06 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984705+0000", 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1052e-05 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147644 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985477+0000", 2024-09-12T07:20:36.009 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.002046291, 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0023589169999999999, 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147644 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985477+0000", 2024-09-12T07:20:36.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985477+0000", 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985478+0000", 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6899999999999998e-07 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985495+0000", 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6966e-05 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985511+0000", 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5965e-05 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985517+0000", 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3180000000000003e-06 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.012 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147676 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986421+0000", 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001103014, 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0014714820000000001, 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.013 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147676 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986421+0000", 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986421+0000", 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986421+0000", 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6899999999999997e-07 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986423+0000", 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0059999999999999e-06 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986431+0000", 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.469e-06 2024-09-12T07:20:36.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986437+0000", 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0719999999999996e-06 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147772 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987262+0000", 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00026182899999999999, 2024-09-12T07:20:36.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00067922700000000002, 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147772 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987262+0000", 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987262+0000", 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987262+0000", 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5900000000000002e-07 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987264+0000", 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.013e-06 2024-09-12T07:20:36.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987271+0000", 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4330000000000001e-06 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987276+0000", 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8829999999999997e-06 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.019 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147549 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.984713+0000", 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0028107929999999998, 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0032767489999999998, 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147549 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984713+0000", 2024-09-12T07:20:36.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984713+0000", 2024-09-12T07:20:36.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984713+0000", 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7700000000000004e-07 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984716+0000", 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.277e-06 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984725+0000", 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0610000000000002e-06 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984730+0000", 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2599999999999996e-06 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147645 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985526+0000", 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0019977850000000002, 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002518967, 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.023 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147645 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985526+0000", 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985526+0000", 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985526+0000", 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.99e-07 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985529+0000", 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3549999999999999e-06 2024-09-12T07:20:36.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985536+0000", 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3950000000000003e-06 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985541+0000", 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2009999999999998e-06 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.026 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147677 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986444+0000", 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0010798699999999999, 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0016499139999999999, 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147677 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986444+0000", 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986444+0000", 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986444+0000", 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8700000000000001e-07 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986446+0000", 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9929999999999998e-06 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986453+0000", 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4609999999999996e-06 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986457+0000", 2024-09-12T07:20:36.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.758e-06 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147773 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987283+0000", 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00024093699999999999, 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00086020200000000002, 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.030 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147773 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987283+0000", 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987283+0000", 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987283+0000", 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3599999999999999e-07 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987285+0000", 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.176e-06 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987292+0000", 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6669999999999999e-06 2024-09-12T07:20:36.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987297+0000", 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8230000000000002e-06 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147550 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.984737+0000", 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0027865009999999998, 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0034555060000000001, 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.033 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147550 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984737+0000", 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984737+0000", 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984738+0000", 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2800000000000002e-07 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984740+0000", 2024-09-12T07:20:36.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3130000000000001e-06 2024-09-12T07:20:36.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984747+0000", 2024-09-12T07:20:36.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6069999999999996e-06 2024-09-12T07:20:36.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984751+0000", 2024-09-12T07:20:36.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7169999999999999e-06 2024-09-12T07:20:36.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988202+0000", 2024-09-12T07:20:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00345102 2024-09-12T07:20:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147646 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985549+0000", 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0019751170000000002, 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0026971690000000001, 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147646 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985549+0000", 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985549+0000", 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985549+0000", 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985551+0000", 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985558+0000", 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8650000000000003e-06 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985564+0000", 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2099999999999998e-06 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.041 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147678 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.041 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986464+0000", 2024-09-12T07:20:36.041 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001059651, 2024-09-12T07:20:36.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001816376, 2024-09-12T07:20:36.041 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.041 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.041 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.041 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.041 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.041 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.041 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147678 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986464+0000", 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986464+0000", 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986465+0000", 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9999999999999998e-07 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986467+0000", 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.001e-06 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986473+0000", 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6789999999999999e-06 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986480+0000", 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6270000000000002e-06 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147774 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987312+0000", 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.000211599, 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000992183, 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.044 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147774 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987312+0000", 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987312+0000", 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987313+0000", 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1900000000000003e-07 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987315+0000", 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2239999999999998e-06 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987321+0000", 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4949999999999996e-06 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987334+0000", 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3032e-05 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.047 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147551 3.1d 3:b88d9c84:::rbd_data.11607d06362.0000000000000005:1d [read 1130496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.984758+0000", 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0027656180000000001, 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035769389999999999, 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147551 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984758+0000", 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984758+0000", 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984759+0000", 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0500000000000004e-07 2024-09-12T07:20:36.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984761+0000", 2024-09-12T07:20:36.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9400000000000001e-06 2024-09-12T07:20:36.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984767+0000", 2024-09-12T07:20:36.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3899999999999998e-06 2024-09-12T07:20:36.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984772+0000", 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9110000000000001e-06 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988297+0000", 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035253279999999999 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988333+0000", 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6106999999999998e-05 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.051 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147647 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985572+0000", 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001951892, 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028042369999999998, 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.052 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147647 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985572+0000", 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985572+0000", 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985572+0000", 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3600000000000004e-07 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985574+0000", 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1110000000000002e-06 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985582+0000", 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1099999999999997e-06 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985594+0000", 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2508e-05 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147679 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986487+0000", 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00103699, 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001927804, 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.055 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.056 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.056 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.056 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.056 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.056 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147679 2024-09-12T07:20:36.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.056 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986487+0000", 2024-09-12T07:20:36.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986487+0000", 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986487+0000", 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9700000000000002e-07 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986493+0000", 2024-09-12T07:20:36.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986500+0000", 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7580000000000003e-06 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986505+0000", 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0549999999999999e-06 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147775 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987343+0000", 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00018106700000000001, 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0011200349999999999, 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147775 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987343+0000", 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987343+0000", 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987343+0000", 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8100000000000003e-07 2024-09-12T07:20:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987345+0000", 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.227e-06 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987352+0000", 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9720000000000003e-06 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987358+0000", 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2299999999999999e-06 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147552 3.1d 3:b88d9c84:::rbd_data.11607d06362.0000000000000005:1d [read 1146880~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.984792+0000", 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0027313720000000001, 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003723255, 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.062 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147552 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984792+0000", 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984792+0000", 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984793+0000", 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984795+0000", 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4049999999999998e-06 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984803+0000", 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6029999999999999e-06 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984808+0000", 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5319999999999997e-06 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988410+0000", 2024-09-12T07:20:36.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0036019469999999999 2024-09-12T07:20:36.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988479+0000", 2024-09-12T07:20:36.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8134999999999997e-05 2024-09-12T07:20:36.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.066 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.066 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147648 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.067 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985604+0000", 2024-09-12T07:20:36.067 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0019200669999999999, 2024-09-12T07:20:36.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029721219999999998, 2024-09-12T07:20:36.067 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.067 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.067 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.067 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.067 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.067 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.067 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147648 2024-09-12T07:20:36.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.067 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985604+0000", 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985604+0000", 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985604+0000", 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8800000000000002e-07 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985606+0000", 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2340000000000001e-06 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985613+0000", 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.968e-06 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985625+0000", 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.186e-05 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147680 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986514+0000", 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0010095950000000001, 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002095282, 2024-09-12T07:20:36.070 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147680 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986514+0000", 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986514+0000", 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986515+0000", 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.63e-07 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986517+0000", 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0990000000000001e-06 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986523+0000", 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3280000000000002e-06 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986528+0000", 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6299999999999997e-06 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147776 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987364+0000", 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.000159309, 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001269065, 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147776 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987364+0000", 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987364+0000", 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987365+0000", 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:20:36.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987367+0000", 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0470000000000001e-06 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987373+0000", 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5390000000000004e-06 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987378+0000", 2024-09-12T07:20:36.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8969999999999999e-06 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147553 3.1d 3:b88d9c84:::rbd_data.11607d06362.0000000000000005:1d [read 1204224~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.984818+0000", 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0027061749999999999, 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0038443399999999999, 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.077 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147553 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984818+0000", 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984818+0000", 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984818+0000", 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9500000000000003e-07 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984820+0000", 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1289999999999999e-06 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984827+0000", 2024-09-12T07:20:36.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7789999999999998e-06 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984832+0000", 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7509999999999999e-06 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988579+0000", 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0037471499999999999 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988644+0000", 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5094999999999999e-05 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147649 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985633+0000", 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0018903259999999999, 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030564099999999999, 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147649 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985633+0000", 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985633+0000", 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985634+0000", 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8899999999999999e-07 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985636+0000", 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.441e-06 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985643+0000", 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0430000000000002e-06 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985650+0000", 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3840000000000002e-06 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.084 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147681 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986536+0000", 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00098810099999999991, 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00217738, 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147681 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986536+0000", 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986536+0000", 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986536+0000", 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5699999999999998e-07 2024-09-12T07:20:36.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986538+0000", 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986544+0000", 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1979999999999998e-06 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986549+0000", 2024-09-12T07:20:36.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4129999999999999e-06 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147777 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987386+0000", 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00013720799999999999, 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001349591, 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.088 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147777 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987386+0000", 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987386+0000", 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987387+0000", 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4499999999999997e-07 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987389+0000", 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.922e-06 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987396+0000", 2024-09-12T07:20:36.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5950000000000004e-06 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987401+0000", 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147554 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.984845+0000", 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0026787130000000001, 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039180889999999996, 2024-09-12T07:20:36.091 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147554 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984845+0000", 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984845+0000", 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984845+0000", 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2200000000000004e-07 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984848+0000", 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5179999999999999e-06 2024-09-12T07:20:36.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984864+0000", 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6359999999999999e-05 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984876+0000", 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1950000000000001e-05 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988761+0000", 2024-09-12T07:20:36.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003885009 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147650 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985658+0000", 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0018661540000000001, 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0031520400000000001, 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.095 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147650 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985658+0000", 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985658+0000", 2024-09-12T07:20:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985658+0000", 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7e-07 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985660+0000", 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4250000000000001e-06 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985667+0000", 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5939999999999999e-06 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985672+0000", 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-06 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147682 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986570+0000", 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00095333099999999997, 2024-09-12T07:20:36.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0022873339999999998, 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147682 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986570+0000", 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986570+0000", 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986571+0000", 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6000000000000004e-07 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986573+0000", 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.526e-06 2024-09-12T07:20:36.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986581+0000", 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1899999999999998e-06 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986586+0000", 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0379999999999999e-06 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147778 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987407+0000", 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00011643599999999999, 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001511594, 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147778 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987407+0000", 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987407+0000", 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987408+0000", 2024-09-12T07:20:36.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987410+0000", 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8670000000000001e-06 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987416+0000", 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3130000000000004e-06 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987421+0000", 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8740000000000003e-06 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147555 3.1d 3:b88d9c84:::rbd_data.11607d06362.0000000000000005:1d [read 1294336~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.984887+0000", 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0026363490000000001, 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040861170000000002, 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.105 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147555 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984887+0000", 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984887+0000", 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984888+0000", 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6499999999999999e-07 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984903+0000", 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5174000000000001e-05 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984911+0000", 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3410000000000003e-06 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.984917+0000", 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5929999999999996e-06 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988939+0000", 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0040221010000000001 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.108 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147651 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985680+0000", 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0018441040000000001, 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0033773940000000001, 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147651 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985680+0000", 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985680+0000", 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985680+0000", 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985683+0000", 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5919999999999999e-06 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985689+0000", 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7599999999999997e-06 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985694+0000", 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1680000000000003e-06 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147683 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986593+0000", 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00093027099999999998, 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002512659, 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147683 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986593+0000", 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986593+0000", 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986594+0000", 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7e-07 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986596+0000", 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4059999999999999e-06 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986603+0000", 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7610000000000002e-06 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986615+0000", 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1697e-05 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.115 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147779 3.16 3.824a7eb6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987428+0000", 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.6162999999999994e-05, 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017305879999999999, 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147779 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987428+0000", 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987428+0000", 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987428+0000", 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0900000000000002e-07 2024-09-12T07:20:36.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987440+0000", 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.222e-05 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987447+0000", 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1049999999999997e-06 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987453+0000", 2024-09-12T07:20:36.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1649999999999997e-06 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147780 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987461+0000", 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 6.2680000000000003e-05, 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001746626, 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.119 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147780 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987461+0000", 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987461+0000", 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987462+0000", 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.68e-07 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987464+0000", 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2290000000000002e-06 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987471+0000", 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8090000000000004e-06 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987476+0000", 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2360000000000004e-06 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.122 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147653 3.5 3:a58f1aab:::rbd_data.11607d06362.0000000000000004:1d [read 712704~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.124 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985731+0000", 2024-09-12T07:20:36.124 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001792739, 2024-09-12T07:20:36.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035123870000000001, 2024-09-12T07:20:36.124 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.124 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.124 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:36.124 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.124 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.124 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147653 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985731+0000", 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985731+0000", 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985731+0000", 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8100000000000003e-07 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985734+0000", 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.283e-06 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985743+0000", 2024-09-12T07:20:36.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9740000000000008e-06 2024-09-12T07:20:36.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985755+0000", 2024-09-12T07:20:36.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2156e-05 2024-09-12T07:20:36.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.989182+0000", 2024-09-12T07:20:36.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0034276250000000001 2024-09-12T07:20:36.127 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.127 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147781 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987491+0000", 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 3.2780999999999997e-05, 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0017815839999999999, 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.128 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147781 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987491+0000", 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987491+0000", 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987491+0000", 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2600000000000002e-07 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987494+0000", 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4119999999999999e-06 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987501+0000", 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9099999999999999e-06 2024-09-12T07:20:36.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987506+0000", 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0390000000000004e-06 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147654 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985817+0000", 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00170685, 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0034822249999999998, 2024-09-12T07:20:36.131 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147654 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985817+0000", 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985817+0000", 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985818+0000", 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7299999999999995e-07 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985853+0000", 2024-09-12T07:20:36.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5948000000000002e-05 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985863+0000", 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2599999999999994e-06 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985876+0000", 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3750999999999999e-05 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147782 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987513+0000", 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 1.0233e-05, 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.001809476, 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.135 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147782 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987513+0000", 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987513+0000", 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987514+0000", 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987522+0000", 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5320000000000006e-06 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987530+0000", 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2219999999999996e-06 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987535+0000", 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2490000000000001e-06 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147655 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985889+0000", 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0016346189999999999, 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0034613119999999998, 2024-09-12T07:20:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147655 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985889+0000", 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985889+0000", 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985890+0000", 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8400000000000004e-07 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985901+0000", 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1121000000000001e-05 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985909+0000", 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8699999999999992e-06 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985915+0000", 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6220000000000003e-06 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147783 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987543+0000", 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9999809, 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0018406710000000001, 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147783 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987543+0000", 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987543+0000", 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987543+0000", 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0500000000000004e-07 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987545+0000", 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9910000000000001e-06 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987552+0000", 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6839999999999999e-06 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987562+0000", 2024-09-12T07:20:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0334e-05 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147656 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985925+0000", 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001598955, 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035107990000000002, 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147656 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985925+0000", 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985925+0000", 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985925+0000", 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1500000000000005e-07 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985927+0000", 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1679999999999998e-06 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985934+0000", 2024-09-12T07:20:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5479999999999998e-06 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985947+0000", 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2663000000000001e-05 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147784 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987570+0000", 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9999537, 2024-09-12T07:20:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019190050000000001, 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147784 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987570+0000", 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987570+0000", 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987571+0000", 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9500000000000002e-07 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987573+0000", 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1090000000000001e-06 2024-09-12T07:20:36.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987580+0000", 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8920000000000003e-06 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987591+0000", 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.117e-05 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147657 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985956+0000", 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001567726, 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003582553, 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147657 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985956+0000", 2024-09-12T07:20:36.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985956+0000", 2024-09-12T07:20:36.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985956+0000", 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7999999999999996e-07 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985959+0000", 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3769999999999999e-06 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985967+0000", 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.824e-06 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985972+0000", 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1109999999999999e-06 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147785 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987600+0000", 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9999242, 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0019890950000000002, 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147785 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987600+0000", 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987600+0000", 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987600+0000", 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.63e-07 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987602+0000", 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.227e-06 2024-09-12T07:20:36.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987609+0000", 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8159999999999996e-06 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987614+0000", 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8400000000000002e-06 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147658 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.985980+0000", 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001543764, 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0036358409999999999, 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147658 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.159 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985980+0000", 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985980+0000", 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985980+0000", 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5799999999999999e-07 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985983+0000", 2024-09-12T07:20:36.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2620000000000001e-06 2024-09-12T07:20:36.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985989+0000", 2024-09-12T07:20:36.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4019999999999999e-06 2024-09-12T07:20:36.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.985994+0000", 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.011e-06 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147786 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987621+0000", 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9999022, 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020213789999999998, 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147786 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987621+0000", 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987621+0000", 2024-09-12T07:20:36.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987622+0000", 2024-09-12T07:20:36.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9299999999999998e-07 2024-09-12T07:20:36.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987624+0000", 2024-09-12T07:20:36.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1679999999999998e-06 2024-09-12T07:20:36.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987631+0000", 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3159999999999999e-06 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987636+0000", 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1839999999999998e-06 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.165 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147659 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986002+0000", 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001521763, 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0036663820000000001, 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147659 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986002+0000", 2024-09-12T07:20:36.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986002+0000", 2024-09-12T07:20:36.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986002+0000", 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986004+0000", 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0980000000000001e-06 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986011+0000", 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5749999999999997e-06 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986018+0000", 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5660000000000003e-06 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147787 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987644+0000", 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9998803, 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020502049999999998, 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.169 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147787 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987644+0000", 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987644+0000", 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987644+0000", 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5499999999999998e-07 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987646+0000", 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1909999999999999e-06 2024-09-12T07:20:36.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987653+0000", 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8140000000000003e-06 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987658+0000", 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9439999999999996e-06 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.172 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147660 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986025+0000", 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001499121, 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0036937850000000002, 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147660 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986025+0000", 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986025+0000", 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986025+0000", 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.44e-07 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986027+0000", 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9250000000000002e-06 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986034+0000", 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9650000000000002e-06 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986039+0000", 2024-09-12T07:20:36.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0950000000000003e-06 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147788 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987676+0000", 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9998479, 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020768129999999998, 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147788 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987676+0000", 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987676+0000", 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987676+0000", 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.13e-07 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987679+0000", 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.328e-06 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987687+0000", 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6649999999999992e-06 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987693+0000", 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4870000000000002e-06 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147661 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986046+0000", 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0014775649999999999, 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0037572809999999999, 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.179 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147661 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986046+0000", 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986046+0000", 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986047+0000", 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0699999999999998e-07 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986049+0000", 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9819999999999998e-06 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986055+0000", 2024-09-12T07:20:36.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5679999999999996e-06 2024-09-12T07:20:36.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986061+0000", 2024-09-12T07:20:36.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0100000000000001e-06 2024-09-12T07:20:36.182 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.182 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.182 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147789 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987701+0000", 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9998231, 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0021554830000000001, 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147789 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.183 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987701+0000", 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987701+0000", 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987701+0000", 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987703+0000", 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.086e-06 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987710+0000", 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6359999999999997e-06 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987715+0000", 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9459999999999997e-06 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147662 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986068+0000", 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001455191, 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0038426469999999998, 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.186 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147662 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986068+0000", 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986068+0000", 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986069+0000", 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4400000000000001e-07 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986071+0000", 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8789999999999999e-06 2024-09-12T07:20:36.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986077+0000", 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6259999999999997e-06 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986082+0000", 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9570000000000001e-06 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147742 3.1c 3:3e72a201:::rbd_data.11607d06362.0000000000000001:1d [read 1097728~8192 out=8192b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986879+0000", 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00064470700000000001, 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003067008, 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147742 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.190 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986879+0000", 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986879+0000", 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986879+0000", 2024-09-12T07:20:36.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986882+0000", 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.514e-06 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986889+0000", 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2010000000000002e-06 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986894+0000", 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0409999999999997e-06 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.989768+0000", 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002873801 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.989838+0000", 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0450999999999993e-05 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.989946+0000", 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010748599999999999 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147790 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987722+0000", 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9998021, 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0022747399999999999, 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.194 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147790 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987722+0000", 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987722+0000", 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987722+0000", 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1899999999999998e-07 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987724+0000", 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9709999999999998e-06 2024-09-12T07:20:36.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987730+0000", 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3820000000000001e-06 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987735+0000", 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9259999999999999e-06 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147663 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986089+0000", 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001434612, 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039364370000000001, 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147663 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986089+0000", 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986089+0000", 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986089+0000", 2024-09-12T07:20:36.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0900000000000002e-07 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986098+0000", 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6689999999999995e-06 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986106+0000", 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0490000000000004e-06 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986112+0000", 2024-09-12T07:20:36.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4020000000000001e-06 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147743 3.1c 3:3e72a201:::rbd_data.11607d06362.0000000000000001:1d [read 1114112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986902+0000", 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00062166000000000001, 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003159242, 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.201 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147743 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986902+0000", 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986902+0000", 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986902+0000", 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4000000000000002e-07 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986905+0000", 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0499999999999999e-06 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986911+0000", 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4609999999999996e-06 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986916+0000", 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.045e-06 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.989965+0000", 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030493450000000002 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.990038+0000", 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2150999999999994e-05 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147791 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987742+0000", 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9997816, 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0023515089999999999, 2024-09-12T07:20:36.205 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147791 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987742+0000", 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987742+0000", 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987742+0000", 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9000000000000002e-07 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987749+0000", 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987822+0000", 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3436999999999995e-05 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987939+0000", 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0001169 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.208 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147664 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986119+0000", 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0014045939999999999, 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039998430000000003, 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147664 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986119+0000", 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986119+0000", 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986119+0000", 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0900000000000002e-07 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986122+0000", 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1380000000000001e-06 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986131+0000", 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9700000000000005e-06 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986136+0000", 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147744 3.1c 3.80454e7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986923+0000", 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00060047400000000004, 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0032321950000000002, 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.212 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147744 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986923+0000", 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986923+0000", 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986924+0000", 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2e-07 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986932+0000", 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4600000000000003e-06 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986940+0000", 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9629999999999998e-06 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986947+0000", 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.533e-06 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.990154+0000", 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0032074849999999999 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.215 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147792 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987968+0000", 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9995556, 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002241664, 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.216 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147792 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987968+0000", 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987968+0000", 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987969+0000", 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3900000000000004e-07 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987972+0000", 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4300000000000002e-06 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987982+0000", 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4809999999999995e-06 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987997+0000", 2024-09-12T07:20:36.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5531000000000001e-05 2024-09-12T07:20:36.219 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.219 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.219 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.219 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147665 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.219 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986143+0000", 2024-09-12T07:20:36.219 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001380434, 2024-09-12T07:20:36.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0041210739999999997, 2024-09-12T07:20:36.219 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147665 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986143+0000", 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986143+0000", 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986144+0000", 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.89e-07 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986146+0000", 2024-09-12T07:20:36.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0870000000000001e-06 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986153+0000", 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9330000000000004e-06 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986164+0000", 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1449e-05 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.222 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147745 3.1c 3.80454e7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986956+0000", 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.000568018, 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0033574099999999999, 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147745 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986956+0000", 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986956+0000", 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986956+0000", 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986958+0000", 2024-09-12T07:20:36.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1619999999999998e-06 2024-09-12T07:20:36.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986965+0000", 2024-09-12T07:20:36.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3899999999999998e-06 2024-09-12T07:20:36.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986970+0000", 2024-09-12T07:20:36.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2789999999999998e-06 2024-09-12T07:20:36.225 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147793 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.988007+0000", 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.999517, 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024280579999999999, 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.226 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147793 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988007+0000", 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988007+0000", 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988007+0000", 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3e-07 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988015+0000", 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3200000000000002e-06 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988022+0000", 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3429999999999998e-06 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988027+0000", 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2299999999999999e-06 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147666 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986172+0000", 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001351348, 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0042895379999999999, 2024-09-12T07:20:36.229 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147666 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986172+0000", 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986172+0000", 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986173+0000", 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4000000000000002e-07 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986175+0000", 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.17e-06 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986181+0000", 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4810000000000003e-06 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986193+0000", 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1289e-05 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.232 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147746 3.1c 3.80454e7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986978+0000", 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00054579200000000002, 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035114579999999999, 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147746 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986978+0000", 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986978+0000", 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986978+0000", 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:20:36.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986981+0000", 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.17e-06 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986987+0000", 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7129999999999999e-06 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986992+0000", 2024-09-12T07:20:36.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8640000000000004e-06 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147794 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.988036+0000", 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9994879, 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0024786029999999998, 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.236 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147794 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988036+0000", 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988036+0000", 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988036+0000", 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4899999999999995e-07 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988038+0000", 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.283e-06 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988045+0000", 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4389999999999997e-06 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988050+0000", 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8720000000000001e-06 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147667 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986201+0000", 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0013226690000000001, 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0043401489999999997, 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147667 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986201+0000", 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986201+0000", 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986201+0000", 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6400000000000003e-07 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986208+0000", 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986215+0000", 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.815e-06 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986220+0000", 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1279999999999999e-06 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.242 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147747 3.1c 3.80454e7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.987016+0000", 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.00050759199999999996, 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003569755, 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147747 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987016+0000", 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987016+0000", 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987017+0000", 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7899999999999997e-07 2024-09-12T07:20:36.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987019+0000", 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.294e-06 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987026+0000", 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3300000000000001e-06 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.987032+0000", 2024-09-12T07:20:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7230000000000001e-06 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147795 3.5 3.d558f1a5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.988068+0000", 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9994559, 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0025674449999999998, 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147795 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988068+0000", 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988068+0000", 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988068+0000", 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3099999999999998e-07 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988071+0000", 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4719999999999998e-06 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988078+0000", 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2119999999999997e-06 2024-09-12T07:20:36.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988083+0000", 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0839999999999999e-06 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147668 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986228+0000", 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0012958399999999999, 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0044612050000000002, 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.249 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147668 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986228+0000", 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986228+0000", 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986228+0000", 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8500000000000002e-07 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986230+0000", 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.114e-06 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986237+0000", 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5019999999999998e-06 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986243+0000", 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1530000000000002e-06 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.252 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147796 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.988090+0000", 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.999434, 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.002648329, 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147796 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988090+0000", 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988090+0000", 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988090+0000", 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8799999999999998e-07 2024-09-12T07:20:36.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988092+0000", 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.77e-06 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988098+0000", 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0079999999999999e-06 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988103+0000", 2024-09-12T07:20:36.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4710000000000001e-06 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147669 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986251+0000", 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0012731559999999999, 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.004538205, 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.256 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147669 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986251+0000", 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986251+0000", 2024-09-12T07:20:36.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986251+0000", 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986253+0000", 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9709999999999998e-06 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986260+0000", 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7259999999999996e-06 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986264+0000", 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8040000000000001e-06 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147797 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.988108+0000", 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9994154, 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0027276459999999998, 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147797 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988108+0000", 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988108+0000", 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988109+0000", 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9800000000000003e-07 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988110+0000", 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4589999999999999e-06 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988114+0000", 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988116+0000", 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5160000000000001e-06 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.262 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147670 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986271+0000", 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0012522500000000001, 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0046017819999999996, 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147670 2024-09-12T07:20:36.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986271+0000", 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986271+0000", 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986272+0000", 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4499999999999997e-07 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986274+0000", 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9360000000000002e-06 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986280+0000", 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3269999999999997e-06 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986285+0000", 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8960000000000002e-06 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147798 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.988120+0000", 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.999404, 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0027948080000000002, 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147798 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988120+0000", 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988120+0000", 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988120+0000", 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2700000000000001e-07 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988121+0000", 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.125e-06 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988124+0000", 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.388e-06 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988128+0000", 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9219999999999999e-06 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147671 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986292+0000", 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001231935, 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0046569469999999998, 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147671 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986292+0000", 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986292+0000", 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986292+0000", 2024-09-12T07:20:36.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8700000000000001e-07 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986294+0000", 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9369999999999998e-06 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986301+0000", 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6540000000000002e-06 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986305+0000", 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7140000000000001e-06 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147799 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.988132+0000", 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9993911, 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028538869999999998, 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.273 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147799 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988132+0000", 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988132+0000", 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988133+0000", 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.35e-07 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988134+0000", 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.263e-06 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988137+0000", 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.399e-06 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988140+0000", 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4830000000000002e-06 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.276 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147672 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986314+0000", 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0012099070000000001, 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0047067230000000003, 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147672 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986314+0000", 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986314+0000", 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986314+0000", 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1800000000000001e-07 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986316+0000", 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8959999999999999e-06 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986323+0000", 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5400000000000001e-06 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986329+0000", 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0510000000000002e-06 2024-09-12T07:20:36.279 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147800 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.988143+0000", 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9993806, 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029150700000000001, 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.280 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147800 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988143+0000", 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988143+0000", 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988144+0000", 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.35e-07 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988145+0000", 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1739999999999999e-06 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988148+0000", 2024-09-12T07:20:36.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.365e-06 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988151+0000", 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4930000000000001e-06 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147673 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986336+0000", 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.001188117, 2024-09-12T07:20:36.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0047560509999999999, 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147673 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986336+0000", 2024-09-12T07:20:36.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986336+0000", 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986336+0000", 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986338+0000", 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9929999999999998e-06 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986344+0000", 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3570000000000003e-06 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986350+0000", 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6380000000000001e-06 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.286 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147801 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.988154+0000", 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9993691, 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029744440000000001, 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147801 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.287 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988154+0000", 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988154+0000", 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988155+0000", 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.35e-07 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988156+0000", 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2130000000000001e-06 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988159+0000", 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4130000000000002e-06 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988167+0000", 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5530000000000004e-06 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147674 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.986357+0000", 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.0011664889999999999, 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0048239160000000001, 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.290 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147674 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986357+0000", 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986357+0000", 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986358+0000", 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1199999999999998e-07 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986368+0000", 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0499000000000001e-05 2024-09-12T07:20:36.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986376+0000", 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0749999999999998e-06 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.986389+0000", 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3181e-05 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:147802 3.1d 3.2139b11d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:35.988171+0000", 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4294967295.9993529, 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030595919999999999, 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 147802 2024-09-12T07:20:36.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988171+0000", 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988171+0000", 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988171+0000", 2024-09-12T07:20:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7800000000000002e-07 2024-09-12T07:20:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988173+0000", 2024-09-12T07:20:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.46e-06 2024-09-12T07:20:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988176+0000", 2024-09-12T07:20:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7730000000000001e-06 2024-09-12T07:20:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:35.988183+0000", 2024-09-12T07:20:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1659999999999999e-06 2024-09-12T07:20:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T07:20:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 87 2024-09-12T07:20:36.297 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:36.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:36.300 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:36.300+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:36.300 INFO:teuthology.orchestra.run.smithi046.stdout:3769 writesame 0x7a7f589 thru 0x7a8c5e1 data_size 0x17(0xd059 bytes) 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:3771 trunc from 0x7a8c5e2 to 0xab622c 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:3772 punch from 0x5c3503 to 0x5cf109, (0xbc06 bytes) 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:3773 write 0xa1ca303 thru 0xa1ca456 (0x154 bytes) 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:3774 writesame 0x841d94b thru 0x8423b05 data_size 0x1(0x61bb bytes) 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:3775 read 0x2640300 thru 0x264221d (0x1f1e bytes) 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:3776 write 0x8cb9af0 thru 0x8cbe04e (0x455f bytes) 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:3778 read 0x1cdaa2f thru 0x1ce530e (0xa8e0 bytes) 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:3779 punch from 0x942ea66 to 0x943c314, (0xd8ae bytes) 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:3780 writesame 0x9c3699a thru 0x9c3a738 data_size 0x5(0x3d9f bytes) 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:3781 read 0x2363fb2 thru 0x2365465 (0x14b4 bytes) 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:36.301 INFO:teuthology.orchestra.run.smithi046.stdout:3783 clone 28 order 24 su 131072 sc 2 2024-09-12T07:20:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:set osd.6 primary-affinity to 0 (802) 2024-09-12T07:20:36.378 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone27 from 0xa1ca457 (overlap 0xab622c) to 0x0 2024-09-12T07:20:36.399 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:36.401+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:36.466 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:36.466 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:36.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:36.501 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:36.503+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:36.582 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:36.582 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:36.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:36.582 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:36.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.582 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.101540513000003, 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:20:36.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:20:36.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:20:36.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:20:36.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:20:36.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.101513633000003, 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:20:36.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.591 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.592 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.592 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:36.592 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.101489612999998, 2024-09-12T07:20:36.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:20:36.592 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.592 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.592 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.592 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.592 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.592 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:20:36.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:20:36.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:20:36.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:20:36.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:20:36.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:20:36.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:20:36.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:20:36.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:20:36.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:20:36.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:20:36.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:20:36.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:20:36.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:20:36.596 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.596 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.101461966999999, 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.597 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:20:36.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:20:36.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.101438504999997, 2024-09-12T07:20:36.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.605 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.101415977999999, 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:36.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.101394175000003, 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.610 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:20:36.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:20:36.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:20:36.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:20:36.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:20:36.614 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.614 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.614 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.615 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.615 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:36.615 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.094760395000002, 2024-09-12T07:20:36.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:20:36.615 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.615 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.615 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.615 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:36.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:20:36.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.094740387000002, 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:20:36.619 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.623 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.088257355000003, 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:20:36.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.088222068, 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.628 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.088199435, 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.633 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:20:36.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.08814538, 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:20:36.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.641 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.088126439, 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:20:36.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.643 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.643 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.643 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:36.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.643 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.643 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:36.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.646 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.088114502000003, 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:20:36.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:20:36.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:20:36.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:20:36.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:20:36.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:20:36.651 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.651 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.651 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.651 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.651 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:36.651 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.088102886999998, 2024-09-12T07:20:36.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:20:36.651 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.651 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:20:36.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.081090279000001, 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:36.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:20:36.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:20:36.659 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.081066563999997, 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.660 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.662 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.080935707000002, 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.664 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:20:36.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 33.117264917, 2024-09-12T07:20:36.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:36.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:20:36.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:20:36.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:36.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:20:36.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:20:36.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:36.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:20:36.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:20:36.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:20:36.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:36.674 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:36.676 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:36.676 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:36.677+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:36.777 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:36.779+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:36.824 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:36.824 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:36.825 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:36.825 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:36.833 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:36.877 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:36.879+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:36.939 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:36.939 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:36.939 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:36.939 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:36.940 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:36.949 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:36.991 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:36.992+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:37.079 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:37.081+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:37.098 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:37.098 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:37.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:37.098 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:37.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.099 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:20:37.099 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:37.099 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 44.569626759000002, 2024-09-12T07:20:37.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:20:37.099 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.099 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:20:37.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.103 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.717829092999999, 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.104 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.717705815999999, 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.108 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:37.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:20:37.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.112 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.717502424999999, 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:37.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:20:37.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:20:37.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.717366597999998, 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.117 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:20:37.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.717039942, 2024-09-12T07:20:37.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.125 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.717001054000001, 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:20:37.129 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.71663418, 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.130 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:20:37.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:20:37.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:20:37.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.716499323999997, 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.134 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:37.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.138 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.716129432999999, 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.715939235, 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.143 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:20:37.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:20:37.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:20:37.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:20:37.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:20:37.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.715719155999999, 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.147 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.151 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:37.152 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.715575491000003, 2024-09-12T07:20:37.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:20:37.152 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.152 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.152 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.152 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.153 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:20:37.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.153 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:37.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:20:37.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.715349318999998, 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.157 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:20:37.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:20:37.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:20:37.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:20:37.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:20:37.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:20:37.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:20:37.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:20:37.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:20:37.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:20:37.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.161 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.715148960999997, 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:20:37.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:20:37.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:20:37.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:20:37.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:20:37.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:20:37.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:20:37.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.714741078000003, 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.166 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:37.170 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.714275196000003, 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:20:37.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:20:37.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.712672986000001, 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.175 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:37.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.712138951, 2024-09-12T07:20:37.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:20:37.180 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.180 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.180 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.180 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.180 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.180 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:20:37.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.180 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:20:37.183 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.455940880999997, 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:37.184 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:20:37.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:37.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:20:37.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:20:37.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:37.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:37.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:37.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:20:37.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:20:37.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.188 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:37.188 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:37.188 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:37.190 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:37.190 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:37.191+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:37.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:20:37.306 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:37.308+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:37.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:20:37.345 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:37.345 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:37.346 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:37.346 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:37.357 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:37.382 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-12T07:20:37.400 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:37.402+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:37.428 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:37.428 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:37.448 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:37.448 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:37.465 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:37.467 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:37.467 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:37.467 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:37.467 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:37.467 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:37.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:37.492 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:37.493+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:37.514 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:37.514 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:37.531 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:37.559 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:37.559 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:37.579 INFO:teuthology.orchestra.run.smithi046.stdout:3784 trunc from 0xa1ca457 to 0xc7651fc 2024-09-12T07:20:37.585 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:37.586 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:37.589 INFO:teuthology.orchestra.run.smithi046.stdout:3786 writesame 0xb3b2c42 thru 0xb3bc062 data_size 0x1(0x9421 bytes) 2024-09-12T07:20:37.593 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:37.594+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:37.613 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:37.614 INFO:teuthology.orchestra.run.smithi046.stdout:3788 write 0x27e9c59 thru 0x27ed6d3 (0x3a7b bytes) 2024-09-12T07:20:37.619 INFO:teuthology.orchestra.run.smithi046.stdout:3790 read 0x7dbf303 thru 0x7dcee00 (0xfafe bytes) 2024-09-12T07:20:37.619 INFO:teuthology.orchestra.run.smithi046.stdout:3791 read 0x27c107a thru 0x27ce518 (0xd49f bytes) 2024-09-12T07:20:37.621 INFO:teuthology.orchestra.run.smithi046.stdout:3792 trunc from 0xc7651fc to 0xabe9c61 2024-09-12T07:20:37.627 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:37.627 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:37.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:37.638 INFO:teuthology.orchestra.run.smithi046.stdout:3794 write 0xb4508fd thru 0xb45f951 (0xf055 bytes) 2024-09-12T07:20:37.640 INFO:teuthology.orchestra.run.smithi046.stdout:3795 read 0x44385b5 thru 0x4443d1f (0xb76b bytes) 2024-09-12T07:20:37.648 INFO:teuthology.orchestra.run.smithi046.stdout:3796 trunc from 0xb45f952 to 0x33c8a0d 2024-09-12T07:20:37.670 INFO:teuthology.orchestra.run.smithi046.stdout:3797 write 0x448fb33 thru 0x449e615 (0xeae3 bytes) 2024-09-12T07:20:37.673 INFO:teuthology.orchestra.run.smithi046.stdout:3798 write 0x293fbd9 thru 0x294d77b (0xdba3 bytes) 2024-09-12T07:20:37.675 INFO:teuthology.orchestra.run.smithi046.stdout:3799 read 0x36f0ddf thru 0x36f345f (0x2681 bytes) 2024-09-12T07:20:37.675 INFO:teuthology.orchestra.run.smithi046.stdout:3800 punch from 0x41ff966 to 0x420c655, (0xccef bytes) 2024-09-12T07:20:37.675 INFO:teuthology.orchestra.run.smithi046.stdout:3801 clone 29 order 24 su 1048576 sc 9 2024-09-12T07:20:37.694 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:37.695+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:37.751 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:37.751 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:37.759 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:37.795 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:37.796+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:37.895 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:37.897+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:37.907 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:37.907 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:37.916 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:37.996 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:37.998+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:38.022 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:38.023 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:38.031 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:38.097 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:38.098+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:38.139 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:38.139 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:38.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:38.232 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:38.234+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:38.296 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:38.296 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:38.299 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:38.301+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:38.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:20:38.396 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone28 from 0x449e616 (overlap 0x33c8a0d) to 0xa5b535 2024-09-12T07:20:38.400 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:38.401+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:38.465 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:38.465 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:38.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:38.514 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:38.516+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:38.537 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-12T07:20:38.557 INFO:teuthology.orchestra.run.smithi046.stdout:3802 write 0xdec2654 thru 0xdec4024 (0x19d1 bytes) 2024-09-12T07:20:38.560 INFO:teuthology.orchestra.run.smithi046.stdout:3803 read 0x16857cc thru 0x168afed (0x5822 bytes) 2024-09-12T07:20:38.577 INFO:teuthology.orchestra.run.smithi046.stdout:3804 trunc from 0xdec4025 to 0x1b96be0 2024-09-12T07:20:38.602 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:38.604+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:38.627 INFO:teuthology.orchestra.run.smithi046.stdout:3805 writesame 0x1ceac16 thru 0x1cf9495 data_size 0x78(0xe880 bytes) 2024-09-12T07:20:38.648 INFO:teuthology.orchestra.run.smithi046.stdout:3807 writesame 0xc71bd94 thru 0xc71fd58 data_size 0x5(0x3fc5 bytes) 2024-09-12T07:20:38.648 INFO:teuthology.orchestra.run.smithi046.stdout:3808 read 0xbe3db8f thru 0xbe41090 (0x3502 bytes) 2024-09-12T07:20:38.649 INFO:teuthology.orchestra.run.smithi046.stdout:3810 read 0x92ade83 thru 0x92ae0ec (0x26a bytes) 2024-09-12T07:20:38.649 INFO:teuthology.orchestra.run.smithi046.stdout:3811 write 0xc5f4856 thru 0xc601aba (0xd265 bytes) 2024-09-12T07:20:38.653 INFO:teuthology.orchestra.run.smithi046.stdout:3812 read 0x96c9170 thru 0x96d1311 (0x81a2 bytes) 2024-09-12T07:20:38.659 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:38.660 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:38.662 INFO:teuthology.orchestra.run.smithi046.stdout:3813 trunc from 0xc71fd59 to 0x5307dc5 2024-09-12T07:20:38.672 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:38.675 INFO:teuthology.orchestra.run.smithi046.stdout:3814 punch from 0x2473bb6 to 0x2482236, (0xe680 bytes) 2024-09-12T07:20:38.676 INFO:teuthology.orchestra.run.smithi046.stdout:3815 read 0x62a0df thru 0x6398f3 (0xf815 bytes) 2024-09-12T07:20:38.676 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:38.676 INFO:teuthology.orchestra.run.smithi046.stdout:3817 write 0x4a8b9c7 thru 0x4a9374d (0x7d87 bytes) 2024-09-12T07:20:38.687 INFO:teuthology.orchestra.run.smithi046.stdout:3818 trunc from 0x5307dc5 to 0xeb2ab08 2024-09-12T07:20:38.699 INFO:teuthology.orchestra.run.smithi046.stdout:3819 trunc from 0xeb2ab08 to 0x8ff9b60 2024-09-12T07:20:38.703 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:38.705+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:38.712 INFO:teuthology.orchestra.run.smithi046.stdout:3820 trunc from 0x8ff9b60 to 0x38704ca 2024-09-12T07:20:38.719 INFO:teuthology.orchestra.run.smithi046.stdout:3821 read 0xf2a696 thru 0xf3694d (0xc2b8 bytes) 2024-09-12T07:20:38.720 INFO:teuthology.orchestra.run.smithi046.stdout:3822 punch from 0x5cee31 to 0x5d6c06, (0x7dd5 bytes) 2024-09-12T07:20:38.720 INFO:teuthology.orchestra.run.smithi046.stdout:3826 read 0x35eb01b thru 0x35f8f1d (0xdf03 bytes) 2024-09-12T07:20:38.720 INFO:teuthology.orchestra.run.smithi046.stdout:3828 punch from 0x1ceb9eb to 0x1cf7c0e, (0xc223 bytes) 2024-09-12T07:20:38.737 INFO:teuthology.orchestra.run.smithi046.stdout:3829 write 0xdc6a144 thru 0xdc6b547 (0x1404 bytes) 2024-09-12T07:20:38.740 INFO:teuthology.orchestra.run.smithi046.stdout:3830 read 0x759e327 thru 0x759f99e (0x1678 bytes) 2024-09-12T07:20:38.740 INFO:teuthology.orchestra.run.smithi046.stdout:3831 read 0x4e34686 thru 0x4e3e4e7 (0x9e62 bytes) 2024-09-12T07:20:38.746 INFO:teuthology.orchestra.run.smithi046.stdout:3833 writesame 0xe4d0e00 thru 0xe4dfabc data_size 0x73(0xecbd bytes) 2024-09-12T07:20:38.747 INFO:teuthology.orchestra.run.smithi046.stdout:3836 write 0x78eaa34 thru 0x78f2492 (0x7a5f bytes) 2024-09-12T07:20:38.748 INFO:teuthology.orchestra.run.smithi046.stdout:3837 punch from 0xdc41c37 to 0xdc45d81, (0x414a bytes) 2024-09-12T07:20:38.748 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:38.762 INFO:teuthology.orchestra.run.smithi046.stdout:3839 trunc from 0xe4dfabd to 0x1df3692 2024-09-12T07:20:38.785 INFO:teuthology.orchestra.run.smithi046.stdout:3840 write 0x435bd32 thru 0x4365341 (0x9610 bytes) 2024-09-12T07:20:38.800 INFO:teuthology.orchestra.run.smithi046.stdout:3841 write 0xabcb2c0 thru 0xabd2ee1 (0x7c22 bytes) 2024-09-12T07:20:38.802 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:38.802 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:38.804 INFO:teuthology.orchestra.run.smithi046.stdout:3842 punch from 0xf771e to 0x103880, (0xc162 bytes) 2024-09-12T07:20:38.805 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:38.806+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:38.805 INFO:teuthology.orchestra.run.smithi046.stdout:3843 read 0x8e7d3a9 thru 0x8e89ecc (0xcb24 bytes) 2024-09-12T07:20:38.805 INFO:teuthology.orchestra.run.smithi046.stdout:3844 write 0x86c24e thru 0x86c4fa (0x2ad bytes) 2024-09-12T07:20:38.807 INFO:teuthology.orchestra.run.smithi046.stdout:3845 punch from 0x8794fb2 to 0x87a4b2d, (0xfb7b bytes) 2024-09-12T07:20:38.807 INFO:teuthology.orchestra.run.smithi046.stdout:3846 writesame 0x3db50d3 thru 0x3db9398 data_size 0x16(0x42c6 bytes) 2024-09-12T07:20:38.808 INFO:teuthology.orchestra.run.smithi046.stdout:3847 read 0x6dd0e54 thru 0x6dde1dd (0xd38a bytes) 2024-09-12T07:20:38.809 INFO:teuthology.orchestra.run.smithi046.stdout:3848 writesame 0x3d93b90 thru 0x3d9b8f4 data_size 0x1(0x7d65 bytes) 2024-09-12T07:20:38.812 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:38.824 INFO:teuthology.orchestra.run.smithi046.stdout:3851 trunc from 0xabd2ee2 to 0xc24d97f 2024-09-12T07:20:38.830 INFO:teuthology.orchestra.run.smithi046.stdout:3853 read 0x53b06fe thru 0x53bbc3c (0xb53f bytes) 2024-09-12T07:20:38.831 INFO:teuthology.orchestra.run.smithi046.stdout:3856 writesame 0x8646bd8 thru 0x8652096 data_size 0x1(0xb4bf bytes) 2024-09-12T07:20:38.863 INFO:teuthology.orchestra.run.smithi046.stdout:3857 trunc from 0xc24d97f to 0x7fcede 2024-09-12T07:20:38.878 INFO:teuthology.orchestra.run.smithi046.stdout:3858 punch from 0x6d00aa to 0x6d8678, (0x85ce bytes) 2024-09-12T07:20:38.879 INFO:teuthology.orchestra.run.smithi046.stdout:3859 punch from 0xd1e6b to 0xd276e, (0x903 bytes) 2024-09-12T07:20:38.879 INFO:teuthology.orchestra.run.smithi046.stdout:3860 punch from 0x763d43 to 0x76823d, (0x44fa bytes) 2024-09-12T07:20:38.885 INFO:teuthology.orchestra.run.smithi046.stdout:3862 write 0x15cec05 thru 0x15d3da1 (0x519d bytes) 2024-09-12T07:20:38.887 INFO:teuthology.orchestra.run.smithi046.stdout:3863 punch from 0x8b091e to 0x8ba062, (0x9744 bytes) 2024-09-12T07:20:38.887 INFO:teuthology.orchestra.run.smithi046.stdout:3864 punch from 0xcd3d9c to 0xce079a, (0xc9fe bytes) 2024-09-12T07:20:38.887 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:38.887 INFO:teuthology.orchestra.run.smithi046.stdout:3866 read 0x8e9088 thru 0x8ecf22 (0x3e9b bytes) 2024-09-12T07:20:38.901 INFO:teuthology.orchestra.run.smithi046.stdout:3868 write 0x8b61dbd thru 0x8b668cb (0x4b0f bytes) 2024-09-12T07:20:38.902 INFO:teuthology.orchestra.run.smithi046.stdout:3869 read 0xa65b5a thru 0xa6c82e (0x6cd5 bytes) 2024-09-12T07:20:38.903 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:38.906 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:38.907+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:38.911 INFO:teuthology.orchestra.run.smithi046.stdout:3872 writesame 0xb9874e5 thru 0xb98cb8f data_size 0x1(0x56ab bytes) 2024-09-12T07:20:38.911 INFO:teuthology.orchestra.run.smithi046.stdout:3873 read 0x3205317 thru 0x321116f (0xbe59 bytes) 2024-09-12T07:20:38.911 INFO:teuthology.orchestra.run.smithi046.stdout:3875 trunc from 0xb98cb90 to 0xbca5c6b 2024-09-12T07:20:38.916 INFO:teuthology.orchestra.run.smithi046.stdout:3878 write 0x5059e14 thru 0x5064690 (0xa87d bytes) 2024-09-12T07:20:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:3880 write 0x8078014 thru 0x8078bd7 (0xbc4 bytes) 2024-09-12T07:20:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:3882 write 0xa869d40 thru 0xa86df6e (0x422f bytes) 2024-09-12T07:20:38.924 INFO:teuthology.orchestra.run.smithi046.stdout:3883 write 0x4fc77ea thru 0x4fcd0a4 (0x58bb bytes) 2024-09-12T07:20:38.925 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:38.938 INFO:teuthology.orchestra.run.smithi046.stdout:3885 trunc from 0xbca5c6b to 0x7f7cdc 2024-09-12T07:20:38.947 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:38.947 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:38.953 INFO:teuthology.orchestra.run.smithi046.stdout:3887 punch from 0x2c14ee to 0x2cc93c, (0xb44e bytes) 2024-09-12T07:20:38.957 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:38.977 INFO:teuthology.orchestra.run.smithi046.stdout:3888 write 0xed9f57c thru 0xeda6f0b (0x7990 bytes) 2024-09-12T07:20:38.979 INFO:teuthology.orchestra.run.smithi046.stdout:3889 writesame 0x6884e9a thru 0x688c76c data_size 0x1(0x78d3 bytes) 2024-09-12T07:20:38.997 INFO:teuthology.orchestra.run.smithi046.stdout:3890 read 0xb3ecc85 thru 0xb3fb480 (0xe7fc bytes) 2024-09-12T07:20:38.997 INFO:teuthology.orchestra.run.smithi046.stdout:3891 read 0xbe1baa2 thru 0xbe20886 (0x4de5 bytes) 2024-09-12T07:20:38.997 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.007 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:39.008+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:39.007 INFO:teuthology.orchestra.run.smithi046.stdout:3894 trunc from 0xeda6f0c to 0x64c58f8 2024-09-12T07:20:39.019 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.019 INFO:teuthology.orchestra.run.smithi046.stdout:3896 write 0x1f34df7 thru 0x1f3aa3b (0x5c45 bytes) 2024-09-12T07:20:39.021 INFO:teuthology.orchestra.run.smithi046.stdout:3897 punch from 0x29d8ca9 to 0x29dbb37, (0x2e8e bytes) 2024-09-12T07:20:39.021 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.021 INFO:teuthology.orchestra.run.smithi046.stdout:3899 write 0x2af30c9 thru 0x2af499f (0x18d7 bytes) 2024-09-12T07:20:39.021 INFO:teuthology.orchestra.run.smithi046.stdout:3900 writesame 0x516f65 thru 0x51ba36 data_size 0x3d(0x4ad2 bytes) 2024-09-12T07:20:39.032 INFO:teuthology.orchestra.run.smithi046.stdout:3901 write 0x94c14d5 thru 0x94ce9a9 (0xd4d5 bytes) 2024-09-12T07:20:39.035 INFO:teuthology.orchestra.run.smithi046.stdout:3902 write 0x242ea68 thru 0x2437791 (0x8d2a bytes) 2024-09-12T07:20:39.037 INFO:teuthology.orchestra.run.smithi046.stdout:3903 read 0x344f1cc thru 0x34561df (0x7014 bytes) 2024-09-12T07:20:39.037 INFO:teuthology.orchestra.run.smithi046.stdout:3905 writesame 0x683427 thru 0x689384 data_size 0x27(0x5f5e bytes) 2024-09-12T07:20:39.038 INFO:teuthology.orchestra.run.smithi046.stdout:3906 read 0x3ac321 thru 0x3af600 (0x32e0 bytes) 2024-09-12T07:20:39.039 INFO:teuthology.orchestra.run.smithi046.stdout:3907 read 0x38f7ec7 thru 0x38f955c (0x1696 bytes) 2024-09-12T07:20:39.039 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.039 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.039 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.039 INFO:teuthology.orchestra.run.smithi046.stdout:3913 read 0x6652038 thru 0x665ad3b (0x8d04 bytes) 2024-09-12T07:20:39.040 INFO:teuthology.orchestra.run.smithi046.stdout:3914 punch from 0x8857a12 to 0x8864c1d, (0xd20b bytes) 2024-09-12T07:20:39.040 INFO:teuthology.orchestra.run.smithi046.stdout:3917 read 0x260715f thru 0x261005d (0x8eff bytes) 2024-09-12T07:20:39.040 INFO:teuthology.orchestra.run.smithi046.stdout:3918 read 0x377102c thru 0x37770fb (0x60d0 bytes) 2024-09-12T07:20:39.041 INFO:teuthology.orchestra.run.smithi046.stdout:3920 read 0x1f56d1d thru 0x1f60a39 (0x9d1d bytes) 2024-09-12T07:20:39.041 INFO:teuthology.orchestra.run.smithi046.stdout:3921 clone 30 order 21 su 524288 sc 12 2024-09-12T07:20:39.096 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:39.096 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:39.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:39.146 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:39.147+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:39.208 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:39.210+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:39.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:39.309 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:39.311+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:39.375 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:39.375 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:39.385 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:39.403 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone29 from 0x94ce9aa (overlap 0x7f7cdc) to 0x593dcd 2024-09-12T07:20:39.410 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:39.411+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:39.490 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-12T07:20:39.501 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:39.501 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:39.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:39.512 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:39.513+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:39.523 INFO:teuthology.orchestra.run.smithi046.stdout:3922 trunc from 0x94ce9aa to 0x8ec429b 2024-09-12T07:20:39.535 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.535 INFO:teuthology.orchestra.run.smithi046.stdout:3925 writesame 0x2c37c14 thru 0x2c4112b data_size 0x8(0x9518 bytes) 2024-09-12T07:20:39.540 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.547 INFO:teuthology.orchestra.run.smithi046.stdout:3927 write 0xa7b75d1 thru 0xa7bd304 (0x5d34 bytes) 2024-09-12T07:20:39.550 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.557 INFO:teuthology.orchestra.run.smithi046.stdout:3929 writesame 0xc2512f1 thru 0xc257b38 data_size 0x2f(0x6848 bytes) 2024-09-12T07:20:39.557 INFO:teuthology.orchestra.run.smithi046.stdout:3930 read 0xa68e2f0 thru 0xa699f74 (0xbc85 bytes) 2024-09-12T07:20:39.557 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.558 INFO:teuthology.orchestra.run.smithi046.stdout:3932 punch from 0x499902e to 0x49a4517, (0xb4e9 bytes) 2024-09-12T07:20:39.558 INFO:teuthology.orchestra.run.smithi046.stdout:3934 read 0xa447cca thru 0xa454124 (0xc45b bytes) 2024-09-12T07:20:39.558 INFO:teuthology.orchestra.run.smithi046.stdout:3936 write 0x1ebb90a thru 0x1ebc883 (0xf7a bytes) 2024-09-12T07:20:39.568 INFO:teuthology.orchestra.run.smithi046.stdout:3937 write 0xe3faaf9 thru 0xe4004eb (0x59f3 bytes) 2024-09-12T07:20:39.572 INFO:teuthology.orchestra.run.smithi046.stdout:3938 write 0x39198da thru 0x39230b7 (0x97de bytes) 2024-09-12T07:20:39.576 INFO:teuthology.orchestra.run.smithi046.stdout:3939 writesame 0x5128307 thru 0x512a4e4 data_size 0x22(0x21de bytes) 2024-09-12T07:20:39.580 INFO:teuthology.orchestra.run.smithi046.stdout:3941 read 0x31e2f87 thru 0x31e3d1b (0xd95 bytes) 2024-09-12T07:20:39.580 INFO:teuthology.orchestra.run.smithi046.stdout:3944 write 0x3174b0c thru 0x317904e (0x4543 bytes) 2024-09-12T07:20:39.584 INFO:teuthology.orchestra.run.smithi046.stdout:3946 writesame 0xbdc2d5c thru 0xbdc5e3f data_size 0x15(0x30e4 bytes) 2024-09-12T07:20:39.586 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.587 INFO:teuthology.orchestra.run.smithi046.stdout:3948 read 0x4c7da51 thru 0x4c81106 (0x36b6 bytes) 2024-09-12T07:20:39.587 INFO:teuthology.orchestra.run.smithi046.stdout:3951 read 0x6fe008f thru 0x6fee1d1 (0xe143 bytes) 2024-09-12T07:20:39.587 INFO:teuthology.orchestra.run.smithi046.stdout:3952 read 0x3437861 thru 0x343f03e (0x77de bytes) 2024-09-12T07:20:39.591 INFO:teuthology.orchestra.run.smithi046.stdout:3953 trunc from 0xe4004ec to 0xb68233a 2024-09-12T07:20:39.599 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:39.602 INFO:teuthology.orchestra.run.smithi046.stdout:3954 read 0x4e5395b thru 0x4e54103 (0x7a9 bytes) 2024-09-12T07:20:39.602 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.602 INFO:teuthology.orchestra.run.smithi046.stdout:3956 punch from 0x2e1dc3d to 0x2e1e1a3, (0x566 bytes) 2024-09-12T07:20:39.603 INFO:teuthology.orchestra.run.smithi046.stdout:3957 read 0x33f8481 thru 0x340673e (0xe2be bytes) 2024-09-12T07:20:39.612 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:39.614+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:39.613 INFO:teuthology.orchestra.run.smithi046.stdout:3958 trunc from 0xb68233a to 0x17e1e35 2024-09-12T07:20:39.625 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:39.625 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:39.629 INFO:teuthology.orchestra.run.smithi046.stdout:3959 trunc from 0x17e1e35 to 0xc31b6 2024-09-12T07:20:39.635 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:39.661 INFO:teuthology.orchestra.run.smithi046.stdout:3961 writesame 0xb7516aa thru 0xb756d90 data_size 0x1(0x56e7 bytes) 2024-09-12T07:20:39.661 INFO:teuthology.orchestra.run.smithi046.stdout:3962 write 0x4fde7b5 thru 0x4feac1b (0xc467 bytes) 2024-09-12T07:20:39.663 INFO:teuthology.orchestra.run.smithi046.stdout:3964 read 0x6eefb23 thru 0x6ef5062 (0x5540 bytes) 2024-09-12T07:20:39.663 INFO:teuthology.orchestra.run.smithi046.stdout:3965 read 0x382d592 thru 0x3839b6f (0xc5de bytes) 2024-09-12T07:20:39.664 INFO:teuthology.orchestra.run.smithi046.stdout:3966 writesame 0x43c993e thru 0x43d2c20 data_size 0x1(0x92e3 bytes) 2024-09-12T07:20:39.680 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.684 INFO:teuthology.orchestra.run.smithi046.stdout:3970 trunc from 0xb756d91 to 0xece806a 2024-09-12T07:20:39.690 INFO:teuthology.orchestra.run.smithi046.stdout:3971 write 0xcf0b60d thru 0xcf186f2 (0xd0e6 bytes) 2024-09-12T07:20:39.693 INFO:teuthology.orchestra.run.smithi046.stdout:3972 write 0xd12431f thru 0xd12fc60 (0xb942 bytes) 2024-09-12T07:20:39.696 INFO:teuthology.orchestra.run.smithi046.stdout:3973 punch from 0x5314bf9 to 0x531b0c1, (0x64c8 bytes) 2024-09-12T07:20:39.713 INFO:teuthology.orchestra.run.smithi046.stdout:3975 trunc from 0xece806a to 0x2a1ae6 2024-09-12T07:20:39.713 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:39.715+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:39.725 INFO:teuthology.orchestra.run.smithi046.stdout:3976 read 0x1c4e25 thru 0x1c5702 (0x8de bytes) 2024-09-12T07:20:39.725 INFO:teuthology.orchestra.run.smithi046.stdout:3979 read 0x1e88f3 thru 0x1f52c5 (0xc9d3 bytes) 2024-09-12T07:20:39.725 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.746 INFO:teuthology.orchestra.run.smithi046.stdout:3984 write 0xb49b34b thru 0xb4a4e48 (0x9afe bytes) 2024-09-12T07:20:39.750 INFO:teuthology.orchestra.run.smithi046.stdout:3985 punch from 0x9398573 to 0x93a4a5e, (0xc4eb bytes) 2024-09-12T07:20:39.750 INFO:teuthology.orchestra.run.smithi046.stdout:3986 read 0x26c8f8 thru 0x27a88c (0xdf95 bytes) 2024-09-12T07:20:39.750 INFO:teuthology.orchestra.run.smithi046.stdout:3987 read 0x22f2a1d thru 0x22f3665 (0xc49 bytes) 2024-09-12T07:20:39.750 INFO:teuthology.orchestra.run.smithi046.stdout:3988 writesame 0x6ce0c0b thru 0x6ce8c0e data_size 0x6(0x8004 bytes) 2024-09-12T07:20:39.754 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:39.766 INFO:teuthology.orchestra.run.smithi046.stdout:3989 trunc from 0xb4a4e49 to 0x2f24d8d 2024-09-12T07:20:39.776 INFO:teuthology.orchestra.run.smithi046.stdout:3990 read 0x8a20d1 thru 0x8a7b80 (0x5ab0 bytes) 2024-09-12T07:20:39.776 INFO:teuthology.orchestra.run.smithi046.stdout:3991 punch from 0x1de7e24 to 0x1dedaeb, (0x5cc7 bytes) 2024-09-12T07:20:39.776 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.785 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:39.786 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:39.787 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:39.787 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:39.790 INFO:teuthology.orchestra.run.smithi046.stdout:3993 trunc from 0x2f24d8d to 0xdd2a4d1 2024-09-12T07:20:39.795 INFO:teuthology.orchestra.run.smithi046.stdout:3994 write 0x10e8a77 thru 0x10f249a (0x9a24 bytes) 2024-09-12T07:20:39.796 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:39.798 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.808 INFO:teuthology.orchestra.run.smithi046.stdout:3998 trunc from 0xdd2a4d1 to 0x6192876 2024-09-12T07:20:39.813 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:39.814 INFO:teuthology.orchestra.run.smithi046.stdout:3999 read 0x353ce8d thru 0x354849e (0xb612 bytes) 2024-09-12T07:20:39.814 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:39.815+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:39.815 INFO:teuthology.orchestra.run.smithi046.stdout:4002 trunc from 0x6192876 to 0x49c1125 2024-09-12T07:20:39.828 INFO:teuthology.orchestra.run.smithi046.stdout:4006 writesame 0x5f72378 thru 0x5f79980 data_size 0x29(0x7609 bytes) 2024-09-12T07:20:39.828 INFO:teuthology.orchestra.run.smithi046.stdout:4007 write 0x5aedd1b thru 0x5afb654 (0xd93a bytes) 2024-09-12T07:20:39.831 INFO:teuthology.orchestra.run.smithi046.stdout:4009 read 0x5d6472f thru 0x5d6f85d (0xb12f bytes) 2024-09-12T07:20:39.832 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.832 INFO:teuthology.orchestra.run.smithi046.stdout:4011 read 0x56f075b thru 0x56f17bd (0x1063 bytes) 2024-09-12T07:20:39.832 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.832 INFO:teuthology.orchestra.run.smithi046.stdout:4014 punch from 0x4a8ccb7 to 0x4a9051e, (0x3867 bytes) 2024-09-12T07:20:39.832 INFO:teuthology.orchestra.run.smithi046.stdout:4015 punch from 0x1e39ee4 to 0x1e46705, (0xc821 bytes) 2024-09-12T07:20:39.832 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:39.832 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:39.837 INFO:teuthology.orchestra.run.smithi046.stdout:4016 writesame 0x691c84a thru 0x6921f89 data_size 0x20(0x5740 bytes) 2024-09-12T07:20:39.837 INFO:teuthology.orchestra.run.smithi046.stdout:4018 read 0x26f5872 thru 0x2703059 (0xd7e8 bytes) 2024-09-12T07:20:39.837 INFO:teuthology.orchestra.run.smithi046.stdout:4019 read 0x66a1a3 thru 0x672789 (0x85e7 bytes) 2024-09-12T07:20:39.840 INFO:teuthology.orchestra.run.smithi046.stdout:4020 trunc from 0x6921f8a to 0x4eaedb8 2024-09-12T07:20:39.850 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:39.851 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:39.854 INFO:teuthology.orchestra.run.smithi046.stdout:4021 writesame 0x73fb2d5 thru 0x74054bd data_size 0x1(0xa1e9 bytes) 2024-09-12T07:20:39.854 INFO:teuthology.orchestra.run.smithi046.stdout:4022 punch from 0x5f65cad to 0x5f68320, (0x2673 bytes) 2024-09-12T07:20:39.854 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.854 INFO:teuthology.orchestra.run.smithi046.stdout:4024 write 0x6cdf953 thru 0x6ce8835 (0x8ee3 bytes) 2024-09-12T07:20:39.857 INFO:teuthology.orchestra.run.smithi046.stdout:4026 read 0x4fb51e8 thru 0x4fb7b39 (0x2952 bytes) 2024-09-12T07:20:39.862 INFO:teuthology.orchestra.run.smithi046.stdout:4027 trunc from 0x74054be to 0x2dd2914 2024-09-12T07:20:39.870 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:39.872 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.872 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.884 INFO:teuthology.orchestra.run.smithi046.stdout:4032 write 0x8413e79 thru 0x841501c (0x11a4 bytes) 2024-09-12T07:20:39.887 INFO:teuthology.orchestra.run.smithi046.stdout:4033 write 0x349e9cd thru 0x349eea3 (0x4d7 bytes) 2024-09-12T07:20:39.890 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:39.890 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:39.890 INFO:teuthology.orchestra.run.smithi046.stdout:4035 read 0x8586f thru 0x8bbab (0x633d bytes) 2024-09-12T07:20:39.891 INFO:teuthology.orchestra.run.smithi046.stdout:4037 punch from 0x19aa1e1 to 0x19b0195, (0x5fb4 bytes) 2024-09-12T07:20:39.891 INFO:teuthology.orchestra.run.smithi046.stdout:4038 write 0x3b0d609 thru 0x3b1b41f (0xde17 bytes) 2024-09-12T07:20:39.896 INFO:teuthology.orchestra.run.smithi046.stdout:4040 trunc from 0x841501d to 0x9a18f95 2024-09-12T07:20:39.906 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:39.906 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:39.906 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:39.906 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:39.908 INFO:teuthology.orchestra.run.smithi046.stdout:4041 trunc from 0x9a18f95 to 0x348c71a 2024-09-12T07:20:39.909 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:39.914 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:39.916+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:39.916 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:39.922 INFO:teuthology.orchestra.run.smithi046.stdout:4042 read 0x970dfa thru 0x9748dd (0x3ae4 bytes) 2024-09-12T07:20:39.923 INFO:teuthology.orchestra.run.smithi046.stdout:4044 read 0x30191e5 thru 0x3019f93 (0xdaf bytes) 2024-09-12T07:20:39.923 INFO:teuthology.orchestra.run.smithi046.stdout:4046 punch from 0xd4be26 to 0xd4e83e, (0x2a18 bytes) 2024-09-12T07:20:39.923 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.923 INFO:teuthology.orchestra.run.smithi046.stdout:4048 punch from 0x993cf6 to 0x9a3aff, (0xfe09 bytes) 2024-09-12T07:20:39.923 INFO:teuthology.orchestra.run.smithi046.stdout:4049 read 0x2574aed thru 0x2579871 (0x4d85 bytes) 2024-09-12T07:20:39.923 INFO:teuthology.orchestra.run.smithi046.stdout:4050 writesame 0x2045136 thru 0x204aabb data_size 0x7(0x5986 bytes) 2024-09-12T07:20:39.926 INFO:teuthology.orchestra.run.smithi046.stdout:4051 read 0x18bbbcd thru 0x18bfb41 (0x3f75 bytes) 2024-09-12T07:20:39.935 INFO:teuthology.orchestra.run.smithi046.stdout:4053 write 0x614d2b1 thru 0x6150d66 (0x3ab6 bytes) 2024-09-12T07:20:39.937 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.949 INFO:teuthology.orchestra.run.smithi046.stdout:4055 write 0xb827643 thru 0xb82c0e8 (0x4aa6 bytes) 2024-09-12T07:20:39.952 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:39.952 INFO:teuthology.orchestra.run.smithi046.stdout:4057 writesame 0x6c7d2d4 thru 0x6c85b2e data_size 0x1(0x885b bytes) 2024-09-12T07:20:39.967 INFO:teuthology.orchestra.run.smithi046.stdout:4059 read 0x346cc04 thru 0x34730e0 (0x64dd bytes) 2024-09-12T07:20:39.967 INFO:teuthology.orchestra.run.smithi046.stdout:4060 punch from 0xb0e1708 to 0xb0e214f, (0xa47 bytes) 2024-09-12T07:20:39.976 INFO:teuthology.orchestra.run.smithi046.stdout:4061 write 0xed3ee0f thru 0xed4906f (0xa261 bytes) 2024-09-12T07:20:39.979 INFO:teuthology.orchestra.run.smithi046.stdout:4063 read 0x7b0ca77 thru 0x7b140f0 (0x767a bytes) 2024-09-12T07:20:39.979 INFO:teuthology.orchestra.run.smithi046.stdout:4064 write 0x13f3e28 thru 0x13fc499 (0x8672 bytes) 2024-09-12T07:20:39.983 INFO:teuthology.orchestra.run.smithi046.stdout:4065 punch from 0xc0669 to 0xc9eac, (0x9843 bytes) 2024-09-12T07:20:39.983 INFO:teuthology.orchestra.run.smithi046.stdout:4067 write 0xa3aded2 thru 0xa3b7ba1 (0x9cd0 bytes) 2024-09-12T07:20:39.989 INFO:teuthology.orchestra.run.smithi046.stdout:4069 trunc from 0xed49070 to 0xbcf428c 2024-09-12T07:20:40.000 INFO:teuthology.orchestra.run.smithi046.stdout:4070 read 0x5a71cd4 thru 0x5a72baa (0xed7 bytes) 2024-09-12T07:20:40.001 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:40.001 INFO:teuthology.orchestra.run.smithi046.stdout:4073 read 0x347e137 thru 0x348ae83 (0xcd4d bytes) 2024-09-12T07:20:40.001 INFO:teuthology.orchestra.run.smithi046.stdout:4074 punch from 0x7aa25ae to 0x7ab120a, (0xec5c bytes) 2024-09-12T07:20:40.001 INFO:teuthology.orchestra.run.smithi046.stdout:4075 writesame 0x68c8f50 thru 0x68cb68a data_size 0xb(0x273b bytes) 2024-09-12T07:20:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:4076 read 0x7f3030e thru 0x7f37ab6 (0x77a9 bytes) 2024-09-12T07:20:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:4078 read 0x1348906 thru 0x134fd5e (0x7459 bytes) 2024-09-12T07:20:40.004 INFO:teuthology.orchestra.run.smithi046.stdout:4081 read 0x72e2099 thru 0x72ee299 (0xc201 bytes) 2024-09-12T07:20:40.009 INFO:teuthology.orchestra.run.smithi046.stdout:4082 trunc from 0xbcf428c to 0x7566f23 2024-09-12T07:20:40.015 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:40.017+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:40.024 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:40.024 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:40.024 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:40.024 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:40.024 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:40.025 INFO:teuthology.orchestra.run.smithi046.stdout:4084 trunc from 0x7566f23 to 0x234fe88 2024-09-12T07:20:40.033 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:40.036 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:40.036 INFO:teuthology.orchestra.run.smithi046.stdout:4086 punch from 0x4a3690 to 0x4aa390, (0x6d00 bytes) 2024-09-12T07:20:40.036 INFO:teuthology.orchestra.run.smithi046.stdout:4088 read 0x1720f42 thru 0x172b720 (0xa7df bytes) 2024-09-12T07:20:40.037 INFO:teuthology.orchestra.run.smithi046.stdout:4090 trunc from 0x234fe88 to 0xbd30ee 2024-09-12T07:20:40.048 INFO:teuthology.orchestra.run.smithi046.stdout:4091 punch from 0x2c7d57 to 0x2d08ff, (0x8ba8 bytes) 2024-09-12T07:20:40.049 INFO:teuthology.orchestra.run.smithi046.stdout:4092 punch from 0xae62ad to 0xaed1b3, (0x6f06 bytes) 2024-09-12T07:20:40.063 INFO:teuthology.orchestra.run.smithi046.stdout:4093 write 0x8143fd0 thru 0x8153522 (0xf553 bytes) 2024-09-12T07:20:40.066 INFO:teuthology.orchestra.run.smithi046.stdout:4094 read 0x3506139 thru 0x3506b6a (0xa32 bytes) 2024-09-12T07:20:40.066 INFO:teuthology.orchestra.run.smithi046.stdout:4095 write 0x379b6b5 thru 0x37a8d44 (0xd690 bytes) 2024-09-12T07:20:40.069 INFO:teuthology.orchestra.run.smithi046.stdout:4096 punch from 0x3cf9795 to 0x3cfcb79, (0x33e4 bytes) 2024-09-12T07:20:40.069 INFO:teuthology.orchestra.run.smithi046.stdout:4098 read 0x5bc6b69 thru 0x5bce803 (0x7c9b bytes) 2024-09-12T07:20:40.069 INFO:teuthology.orchestra.run.smithi046.stdout:4099 read 0x6b7c4d2 thru 0x6b80c00 (0x472f bytes) 2024-09-12T07:20:40.069 INFO:teuthology.orchestra.run.smithi046.stdout:4100 read 0x213004e thru 0x2131eea (0x1e9d bytes) 2024-09-12T07:20:40.081 INFO:teuthology.orchestra.run.smithi046.stdout:4101 writesame 0xdb282b5 thru 0xdb33b12 data_size 0x1(0xb85e bytes) 2024-09-12T07:20:40.081 INFO:teuthology.orchestra.run.smithi046.stdout:4102 read 0xa0b1f91 thru 0xa0b2776 (0x7e6 bytes) 2024-09-12T07:20:40.081 INFO:teuthology.orchestra.run.smithi046.stdout:4104 read 0x336ff7f thru 0x3371dd1 (0x1e53 bytes) 2024-09-12T07:20:40.082 INFO:teuthology.orchestra.run.smithi046.stdout:4105 write 0x6fb0440 thru 0x6fb17b6 (0x1377 bytes) 2024-09-12T07:20:40.084 INFO:teuthology.orchestra.run.smithi046.stdout:4106 writesame 0x785b30c thru 0x785d7ce data_size 0x1(0x24c3 bytes) 2024-09-12T07:20:40.090 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:40.090 INFO:teuthology.orchestra.run.smithi046.stdout:4108 clone 31 order 22 su 32768 sc 3 2024-09-12T07:20:40.116 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:40.118+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:40.182 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.701360561999998, 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:20:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:20:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.701333681999998, 2024-09-12T07:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.701309662, 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:20:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:20:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:20:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:20:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:20:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:20:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.701282016, 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:20:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:20:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:20:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:20:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:20:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:20:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:20:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.701258553999999, 2024-09-12T07:20:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:20:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:20:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:20:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:20:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.701236027, 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:20:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:20:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:20:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:20:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:20:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:20:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:20:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:20:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:20:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:20:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:20:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:20:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.701214223999997, 2024-09-12T07:20:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:20:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:20:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:20:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:20:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:20:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:20:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:20:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:20:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:20:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:20:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:20:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.694580444000003, 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:20:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:20:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:20:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:20:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:20:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.694560436000003, 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:20:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:20:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:20:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:20:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:20:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:20:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:20:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:20:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.688077403999998, 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.688042117000002, 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.688019484000002, 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.687965429000002, 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:20:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:20:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:20:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:20:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.687946488000001, 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:20:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:20:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.687934550999998, 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:20:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:20:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:20:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:20:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:20:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:20:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:20:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:20:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:20:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:20:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:20:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:20:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:20:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:20:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.687922936, 2024-09-12T07:20:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:20:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:20:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:20:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:20:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:20:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:20:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:20:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:20:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:20:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:20:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:20:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:20:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:20:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:20:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:20:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:20:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.680910328000003, 2024-09-12T07:20:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:20:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:20:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:20:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:20:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:20:40.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:20:40.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:20:40.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:20:40.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:20:40.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.680886612999998, 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.275 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:20:40.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:20:40.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:20:40.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:20:40.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:20:40.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:20:40.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:20:40.279 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.680755756000003, 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.280 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.281 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:20:40.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.281 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:40.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 36.717084966000002, 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:20:40.284 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.285 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.285 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:40.285 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.285 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.285 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.285 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:20:40.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.285 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:40.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:40.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:20:40.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:20:40.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:20:40.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:20:40.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:20:40.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:20:40.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:20:40.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:20:40.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:20:40.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:20:40.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:20:40.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:20:40.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:20:40.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:20:40.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:20:40.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:40.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:20:40.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:20:40.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:20:40.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:20:40.294 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.294 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.294 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.294 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.294 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.294 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:40.295 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:40.296 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:40.297+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:40.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:20:40.396 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:40.397+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:40.423 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone30 from 0xdb33b13 (overlap 0xc31b6) to 0x88932 2024-09-12T07:20:40.448 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:40.448 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:40.448 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:40.449 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:40.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:40.498 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:40.500+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:40.523 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-12T07:20:40.535 INFO:teuthology.orchestra.run.smithi046.stdout:4109 write 0x41e23a0 thru 0x41e5f6f (0x3bd0 bytes) 2024-09-12T07:20:40.545 INFO:teuthology.orchestra.run.smithi046.stdout:4110 write 0x11f4200 thru 0x11ff656 (0xb457 bytes) 2024-09-12T07:20:40.555 INFO:teuthology.orchestra.run.smithi046.stdout:4111 trunc from 0xdb33b13 to 0xc513407 2024-09-12T07:20:40.565 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:40.565 INFO:teuthology.orchestra.run.smithi046.stdout:4114 punch from 0x829f85e to 0x82a6a46, (0x71e8 bytes) 2024-09-12T07:20:40.565 INFO:teuthology.orchestra.run.smithi046.stdout:4115 write 0x5baeb5c thru 0x5bba4fa (0xb99f bytes) 2024-09-12T07:20:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:4118 punch from 0x1505902 to 0x150f29c, (0x999a bytes) 2024-09-12T07:20:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:4121 read 0xc418ddc thru 0xc41954e (0x773 bytes) 2024-09-12T07:20:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:4122 trunc from 0xc513407 to 0xdb54458 2024-09-12T07:20:40.586 INFO:teuthology.orchestra.run.smithi046.stdout:4124 trunc from 0xdb54458 to 0x56ee4a1 2024-09-12T07:20:40.597 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:40.599+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:40.603 INFO:teuthology.orchestra.run.smithi046.stdout:4125 writesame 0x5f84cc6 thru 0x5f94a1d data_size 0x4(0xfd58 bytes) 2024-09-12T07:20:40.605 INFO:teuthology.orchestra.run.smithi046.stdout:4126 punch from 0x4e3c2ba to 0x4e45012, (0x8d58 bytes) 2024-09-12T07:20:40.605 INFO:teuthology.orchestra.run.smithi046.stdout:4127 read 0x12686f6 thru 0x1269f4a (0x1855 bytes) 2024-09-12T07:20:40.606 INFO:teuthology.orchestra.run.smithi046.stdout:4128 write 0x5734496 thru 0x57422d9 (0xde44 bytes) 2024-09-12T07:20:40.607 INFO:teuthology.orchestra.run.smithi046.stdout:4129 punch from 0x34402b3 to 0x3447015, (0x6d62 bytes) 2024-09-12T07:20:40.607 INFO:teuthology.orchestra.run.smithi046.stdout:4130 read 0x510fec0 thru 0x511243f (0x2580 bytes) 2024-09-12T07:20:40.607 INFO:teuthology.orchestra.run.smithi046.stdout:4131 writesame 0x1ef78af thru 0x1ef7ada data_size 0x2(0x22c bytes) 2024-09-12T07:20:40.611 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:40.611 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:40.611 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:40.611 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:40.611 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:40.619 INFO:teuthology.orchestra.run.smithi046.stdout:4133 trunc from 0x5f94a1e to 0xd711d4b 2024-09-12T07:20:40.620 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:40.632 INFO:teuthology.orchestra.run.smithi046.stdout:4134 trunc from 0xd711d4b to 0x650c679 2024-09-12T07:20:40.648 INFO:teuthology.orchestra.run.smithi046.stdout:4138 writesame 0xb250baf thru 0xb252f1c data_size 0x5(0x236e bytes) 2024-09-12T07:20:40.650 INFO:teuthology.orchestra.run.smithi046.stdout:4141 trunc from 0xb252f1d to 0xd24b0f9 2024-09-12T07:20:40.656 INFO:teuthology.orchestra.run.smithi046.stdout:4142 punch from 0x90d7e92 to 0x90e1d7e, (0x9eec bytes) 2024-09-12T07:20:40.656 INFO:teuthology.orchestra.run.smithi046.stdout:4144 write 0x90476b thru 0x90fd65 (0xb5fb bytes) 2024-09-12T07:20:40.679 INFO:teuthology.orchestra.run.smithi046.stdout:4146 trunc from 0xd24b0f9 to 0x58bc6c4 2024-09-12T07:20:40.698 INFO:teuthology.orchestra.run.smithi046.stdout:4147 write 0xbc2f2a2 thru 0xbc344e2 (0x5241 bytes) 2024-09-12T07:20:40.699 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:40.700+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:40.704 INFO:teuthology.orchestra.run.smithi046.stdout:4148 trunc from 0xbc344e3 to 0x99d779b 2024-09-12T07:20:40.724 INFO:teuthology.orchestra.run.smithi046.stdout:4149 trunc from 0x99d779b to 0x6ab3b1 2024-09-12T07:20:40.745 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:40.746 INFO:teuthology.orchestra.run.smithi046.stdout:4152 punch from 0x10c542 to 0x116d02, (0xa7c0 bytes) 2024-09-12T07:20:40.759 INFO:teuthology.orchestra.run.smithi046.stdout:4153 writesame 0x80608b1 thru 0x806d9e0 data_size 0x8(0xd130 bytes) 2024-09-12T07:20:40.761 INFO:teuthology.orchestra.run.smithi046.stdout:4154 trunc from 0x806d9e1 to 0x9f020e2 2024-09-12T07:20:40.766 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:40.766 INFO:teuthology.orchestra.run.smithi046.stdout:4156 writesame 0x187e67d thru 0x18892ec data_size 0x59(0xac70 bytes) 2024-09-12T07:20:40.770 INFO:teuthology.orchestra.run.smithi046.stdout:4157 read 0x9be6e0c thru 0x9be8968 (0x1b5d bytes) 2024-09-12T07:20:40.771 INFO:teuthology.orchestra.run.smithi046.stdout:4158 write 0x1242e22 thru 0x124d94f (0xab2e bytes) 2024-09-12T07:20:40.774 INFO:teuthology.orchestra.run.smithi046.stdout:4162 writesame 0x2e5422b thru 0x2e60a20 data_size 0x5(0xc7f6 bytes) 2024-09-12T07:20:40.775 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:40.775 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:40.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:40.775 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:40.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.775 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 48.246656846, 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:20:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:20:40.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:20:40.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:20:40.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:20:40.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:20:40.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.394859179999997, 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.781 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.785 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.394735902999997, 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:40.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:20:40.789 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.394532511999998, 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.790 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:20:40.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.394396684999997, 2024-09-12T07:20:40.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:20:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.394070028999998, 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:40.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:20:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:20:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:20:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:20:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:20:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:20:40.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:20:40.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:20:40.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:20:40.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.803 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.394031140999999, 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:20:40.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.393664266999998, 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.808 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.809 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.809 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.809 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.809 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:20:40.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.809 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:40.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:20:40.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:20:40.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.393529411000003, 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:20:40.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:20:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:20:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:20:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.393159519999998, 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.817 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:20:40.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.821 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.392969321999999, 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:40.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:40.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:20:40.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.392749242999997, 2024-09-12T07:20:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:20:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.830 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.392605578000001, 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.831 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:20:40.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.392379406000003, 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.835 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:40.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:20:40.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.392179048000003, 2024-09-12T07:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:40.843 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.391771165000002, 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:40.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:40.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:20:40.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:20:40.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:20:40.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.391305283000001, 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:20:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.389703073, 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.855 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:40.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:20:40.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:20:40.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:20:40.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.859 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.389169037999999, 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.860 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.132970968000002, 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:40.864 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:40.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:40.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:20:40.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:20:40.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:40.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:20:40.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:20:40.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:40.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:20:40.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.868 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:40.869 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:40.869 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:40.871 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:4165 read 0x2083e20 thru 0x2090e99 (0xd07a bytes) 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:4166 writesame 0x345b82f thru 0x346033d data_size 0x3f(0x4b0f bytes) 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:4167 read 0x86b3b72 thru 0x86b857b (0x4a0a bytes) 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:4168 read 0x26200a4 thru 0x2625d75 (0x5cd2 bytes) 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:4169 write 0x112b79b thru 0x112c560 (0xdc6 bytes) 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:4170 read 0x65a9ee1 thru 0x65b1c62 (0x7d82 bytes) 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:4171 trunc from 0x9f020e2 to 0x9a5f35 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:4173 read 0x51b9b9 thru 0x52762d (0xbc75 bytes) 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:4175 read 0x5cf0eb thru 0x5d78e1 (0x87f7 bytes) 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:4176 write 0x99b8861 thru 0x99bf7ce (0x6f6e bytes) 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:4177 read 0x36c556c thru 0x36d2538 (0xcfcd bytes) 2024-09-12T07:20:40.873 INFO:teuthology.orchestra.run.smithi046.stdout:4179 read 0x60cf0e7 thru 0x60d92f9 (0xa213 bytes) 2024-09-12T07:20:40.874 INFO:teuthology.orchestra.run.smithi046.stdout:4183 read 0x596f2cd thru 0x597d5a5 (0xe2d9 bytes) 2024-09-12T07:20:40.874 INFO:teuthology.orchestra.run.smithi046.stdout:4185 writesame 0xb75ad14 thru 0xb75b65c data_size 0x1(0x949 bytes) 2024-09-12T07:20:40.874 INFO:teuthology.orchestra.run.smithi046.stdout:4189 writesame 0x7ab1aa9 thru 0x7ab1d16 data_size 0x1(0x26e bytes) 2024-09-12T07:20:40.874 INFO:teuthology.orchestra.run.smithi046.stdout:4190 punch from 0x6f27817 to 0x6f342ed, (0xcad6 bytes) 2024-09-12T07:20:40.874 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:40.874 INFO:teuthology.orchestra.run.smithi046.stdout:4192 punch from 0x1423d38 to 0x1426578, (0x2840 bytes) 2024-09-12T07:20:40.874 INFO:teuthology.orchestra.run.smithi046.stdout:4193 read 0x3d331c3 thru 0x3d40d2d (0xdb6b bytes) 2024-09-12T07:20:40.874 INFO:teuthology.orchestra.run.smithi046.stdout:4194 trunc from 0xb75b65d to 0xd5df6b4 2024-09-12T07:20:40.874 INFO:teuthology.orchestra.run.smithi046.stdout:4195 read 0x65db94 thru 0x66453d (0x69aa bytes) 2024-09-12T07:20:40.874 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:40.875 INFO:teuthology.orchestra.run.smithi046.stdout:4197 writesame 0x47081d thru 0x470aed data_size 0x1(0x2d1 bytes) 2024-09-12T07:20:40.875 INFO:teuthology.orchestra.run.smithi046.stdout:4199 trunc from 0xd5df6b4 to 0x7158b28 2024-09-12T07:20:40.875 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:40.875 INFO:teuthology.orchestra.run.smithi046.stdout:4201 read 0x4f31693 thru 0x4f38b73 (0x74e1 bytes) 2024-09-12T07:20:40.875 INFO:teuthology.orchestra.run.smithi046.stdout:4203 read 0x1a59742 thru 0x1a65ce8 (0xc5a7 bytes) 2024-09-12T07:20:40.875 INFO:teuthology.orchestra.run.smithi046.stdout:4204 punch from 0x46dc7e5 to 0x46e02ba, (0x3ad5 bytes) 2024-09-12T07:20:40.875 INFO:teuthology.orchestra.run.smithi046.stdout:4207 write 0x4ca8f91 thru 0x4caae1d (0x1e8d bytes) 2024-09-12T07:20:40.875 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:40.871+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:40.880 INFO:teuthology.orchestra.run.smithi046.stdout:4208 write 0xdbfc401 thru 0xdc09f67 (0xdb67 bytes) 2024-09-12T07:20:40.897 INFO:teuthology.orchestra.run.smithi046.stdout:4209 trunc from 0xdc09f68 to 0x2a96342 2024-09-12T07:20:40.914 INFO:teuthology.orchestra.run.smithi046.stdout:4210 trunc from 0x2a96342 to 0x9d717e1 2024-09-12T07:20:40.926 INFO:teuthology.orchestra.run.smithi046.stdout:4211 writesame 0xb6b896b thru 0xb6c2ea3 data_size 0x17(0xa539 bytes) 2024-09-12T07:20:40.927 INFO:teuthology.orchestra.run.smithi046.stdout:4212 write 0x2759d80 thru 0x2768ac1 (0xed42 bytes) 2024-09-12T07:20:40.931 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:40.932 INFO:teuthology.orchestra.run.smithi046.stdout:4215 trunc from 0xb6c2ea4 to 0xc123a3e 2024-09-12T07:20:40.938 INFO:teuthology.orchestra.run.smithi046.stdout:4216 trunc from 0xc123a3e to 0xaa6108d 2024-09-12T07:20:40.943 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:40.943 INFO:teuthology.orchestra.run.smithi046.stdout:4218 read 0x2088885 thru 0x2088c32 (0x3ae bytes) 2024-09-12T07:20:40.944 INFO:teuthology.orchestra.run.smithi046.stdout:4219 read 0x2cb3109 thru 0x2cb5e1d (0x2d15 bytes) 2024-09-12T07:20:40.944 INFO:teuthology.orchestra.run.smithi046.stdout:4220 write 0x1c8455 thru 0x1ccbb6 (0x4762 bytes) 2024-09-12T07:20:40.946 INFO:teuthology.orchestra.run.smithi046.stdout:4221 writesame 0x709f5ed thru 0x70a09ee data_size 0xd(0x1402 bytes) 2024-09-12T07:20:40.950 INFO:teuthology.orchestra.run.smithi046.stdout:4222 trunc from 0xaa6108d to 0xa6958cb 2024-09-12T07:20:40.954 INFO:teuthology.orchestra.run.smithi046.stdout:4223 writesame 0x7e10014 thru 0x7e11b73 data_size 0x8(0x1b60 bytes) 2024-09-12T07:20:40.959 INFO:teuthology.orchestra.run.smithi046.stdout:4225 trunc from 0xa6958cb to 0xbd70132 2024-09-12T07:20:40.965 INFO:teuthology.orchestra.run.smithi046.stdout:4226 trunc from 0xbd70132 to 0xc4aa1dd 2024-09-12T07:20:40.970 INFO:teuthology.orchestra.run.smithi046.stdout:4227 read 0x16f9a12 thru 0x1700c2c (0x721b bytes) 2024-09-12T07:20:40.970 INFO:teuthology.orchestra.run.smithi046.stdout:4228 read 0x6a8e0ab thru 0x6a95f95 (0x7eeb bytes) 2024-09-12T07:20:40.970 INFO:teuthology.orchestra.run.smithi046.stdout:4229 clone 32 order 25 su 262144 sc 13 2024-09-12T07:20:40.971 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:40.972+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:40.985 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:40.986 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:40.986 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:40.986 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:40.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:41.078 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:41.080+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:41.142 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:41.142 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:41.142 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:41.143 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:41.143 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:41.150 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:41.191 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:41.193+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:41.272 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:41.274+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:41.299 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751591826000002, 2024-09-12T07:20:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:20:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:20:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:20:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:20:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:20:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:20:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:20:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751571288999997, 2024-09-12T07:20:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:20:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:20:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751551286000002, 2024-09-12T07:20:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:20:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751531337000003, 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:20:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751510469000003, 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:20:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:20:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:20:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:20:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:20:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:20:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:20:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751488768000002, 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:20:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751468328000001, 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:20:41.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751440606999999, 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:41.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:20:41.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:20:41.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:20:41.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:20:41.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.337 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751417672000002, 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:20:41.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751397372, 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.342 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.343 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.343 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.343 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.343 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:20:41.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.343 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:41.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:41.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:41.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:41.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751377095999999, 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.351 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751356936999997, 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:20:41.355 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751219739, 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:20:41.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:20:41.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:20:41.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:20:41.360 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.360 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.360 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.360 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.360 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:41.361 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.751197091000002, 2024-09-12T07:20:41.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:20:41.361 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.361 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.361 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.361 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.361 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.361 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:20:41.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.361 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:20:41.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:20:41.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:20:41.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68088 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 393216~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.744754331999999, 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10346986399999999, 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.365 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68088 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555923+0000", 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.891e-06 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555929+0000", 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0549999999999996e-06 2024-09-12T07:20:41.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555934+0000", 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6419999999999998e-06 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659060+0000", 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103125862 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659139+0000", 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9450000000000007e-05 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659391+0000", 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025158400000000001 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:41.369 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.744735104999997, 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:20:41.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:20:41.373 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.744505468, 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.374 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.375 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.375 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:20:41.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.375 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:41.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:41.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:20:41.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:20:41.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:20:41.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.378 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.737581777000003, 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.379 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:20:41.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:20:41.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:20:41.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:20:41.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:20:41.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:20:41.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:20:41.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:20:41.383 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.383 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.383 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.737222193999997, 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:20:41.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:20:41.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:20:41.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:20:41.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:20:41.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:20:41.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:20:41.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:20:41.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:20:41.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:20:41.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:20:41.388 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.388 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.388 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.737142208000002, 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:20:41.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:41.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:20:41.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:41.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:41.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:20:41.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:20:41.393 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.393 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.393 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.393 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:41.393 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:41.393 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:41.393 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:20:41.393 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:20:41.393 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-12T07:20:41.394 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T07:20:41.396 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:20:41.396 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:41.396+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:41.396 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:41.433 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone31 from 0xc4aa1dd (overlap 0x6ab3b1) to 0x40056a 2024-09-12T07:20:41.496 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:41.498+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:41.549 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-12T07:20:41.553 INFO:teuthology.orchestra.run.smithi046.stdout:4230 read 0x1344f81 thru 0x1354d85 (0xfe05 bytes) 2024-09-12T07:20:41.554 INFO:teuthology.orchestra.run.smithi046.stdout:4231 writesame 0x406ae93 thru 0x4071c93 data_size 0x3f(0x6e01 bytes) 2024-09-12T07:20:41.559 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:41.559 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:41.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.559 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:152441 3.17 3:efd96fda:::rbd_data.116020aa56a5.0000000000000001:22 [sparse-read 884736~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-12T07:20:41.559 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:41.560777+0000", 2024-09-12T07:20:41.559 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.00010611, 2024-09-12T07:20:41.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012421399999999999, 2024-09-12T07:20:41.559 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-12T07:20:41.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 152441 2024-09-12T07:20:41.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.561 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560777+0000", 2024-09-12T07:20:41.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560777+0000", 2024-09-12T07:20:41.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560778+0000", 2024-09-12T07:20:41.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1399999999999996e-07 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560783+0000", 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4970000000000003e-06 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560795+0000", 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2622999999999999e-05 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560807+0000", 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2177999999999999e-05 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560866+0000", 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.842e-05 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:152442 3.17 3.5bf69bf7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-12T07:20:41.564 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:41.560834+0000", 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.8955999999999997e-05, 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101358, 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 152442 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560834+0000", 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560834+0000", 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560835+0000", 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8699999999999995e-07 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560839+0000", 2024-09-12T07:20:41.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6519999999999999e-06 2024-09-12T07:20:41.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560848+0000", 2024-09-12T07:20:41.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0690000000000008e-06 2024-09-12T07:20:41.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560854+0000", 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:152443 3.17 3.5bf69bf7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:41.560867+0000", 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.6676000000000001e-05, 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119218, 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 152443 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560867+0000", 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560867+0000", 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560867+0000", 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.15e-07 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560870+0000", 2024-09-12T07:20:41.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7999999999999999e-06 2024-09-12T07:20:41.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560899+0000", 2024-09-12T07:20:41.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8663999999999999e-05 2024-09-12T07:20:41.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560904+0000", 2024-09-12T07:20:41.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6239999999999999e-06 2024-09-12T07:20:41.571 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:152446 3.12 3:49c740cd:::rbd_header.1160e03d6713:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:41.560917+0000", 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9999661, 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000121915, 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.572 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 152446 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560917+0000", 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560917+0000", 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560918+0000", 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2099999999999996e-07 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560922+0000", 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6349999999999999e-06 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560929+0000", 2024-09-12T07:20:41.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8580000000000002e-06 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.560932+0000", 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3230000000000002e-06 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:41.561010+0000", 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7681000000000004e-05 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.576 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:41.576 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 4 2024-09-12T07:20:41.576 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:41.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:41.578 INFO:teuthology.orchestra.run.smithi046.stdout:4233 writesame 0xd70f020 thru 0xd718f70 data_size 0x5(0x9f51 bytes) 2024-09-12T07:20:41.578 INFO:teuthology.orchestra.run.smithi046.stdout:4234 write 0x6755455 thru 0x675fdbf (0xa96b bytes) 2024-09-12T07:20:41.578 INFO:teuthology.orchestra.run.smithi046.stdout:4235 read 0xc4e2529 thru 0xc4f0324 (0xddfc bytes) 2024-09-12T07:20:41.578 INFO:teuthology.orchestra.run.smithi046.stdout:4236 punch from 0xb0ae7aa to 0xb0b26b5, (0x3f0b bytes) 2024-09-12T07:20:41.578 INFO:teuthology.orchestra.run.smithi046.stdout:4237 read 0xa8bc632 thru 0xa8c02a4 (0x3c73 bytes) 2024-09-12T07:20:41.578 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:41.578 INFO:teuthology.orchestra.run.smithi046.stdout:4240 punch from 0x631483c to 0x631e7b4, (0x9f78 bytes) 2024-09-12T07:20:41.590 INFO:teuthology.orchestra.run.smithi046.stdout:4241 trunc from 0xd718f71 to 0x1f12dfb 2024-09-12T07:20:41.597 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:41.599+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:41.622 INFO:teuthology.orchestra.run.smithi046.stdout:4242 read 0xff28f1 thru 0xffe3b2 (0xbac2 bytes) 2024-09-12T07:20:41.622 INFO:teuthology.orchestra.run.smithi046.stdout:4243 clone 33 order 22 su 262144 sc 10 2024-09-12T07:20:41.698 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:41.700+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:41.704 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:41.704 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:41.704 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:41.705 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:41.705 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:41.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:20:41.775 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 80, 2024-09-12T07:20:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "8412ed58-ac5e-4549-b523-1a58cf9f9429", 2024-09-12T07:20:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "created": "2024-09-12T07:19:10.328057+0000", 2024-09-12T07:20:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-12T07:20:41.338641+0000", 2024-09-12T07:20:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "last_up_change": "2024-09-12T07:19:22.177473+0000", 2024-09-12T07:20:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "last_in_change": "2024-09-12T07:20:06.162815+0000", 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_num": 5801984, 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_set": [ 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "noscrub", 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "pglog_hardlimit", 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "purged_snapdirs", 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_deletes", 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "sortbitwise" 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_version": 5, 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T07:20:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_snapshot": "", 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_max": 3, 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd": 8, 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "require_min_compat_client": "luminous", 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "min_compat_client": "jewel", 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "require_osd_release": "squid", 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_crimson": false, 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "pools": [ 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 1, 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": ".mgr", 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "create_time": "2024-09-12T07:19:23.507151+0000", 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T07:20:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_names": "hashpspool", 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type": 1, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 2, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": 1, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_rule": 0, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_count": 0, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_target": 0, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "is_stretch_pool": false, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "object_hash": 2, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscale_mode": "off", 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": 1, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num": 1, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num_target": 1, 2024-09-12T07:20:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_target": 1, 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_pending": 1, 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "last_pg_merge_meta": { 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "source_pgid": "0.0", 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "ready_epoch": 0, 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_started": 0, 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_clean": 0, 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "source_version": "0'0", 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "target_version": "0'0" 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "last_change": "18", 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend": "0", 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "auid": 0, 2024-09-12T07:20:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_mode": "selfmanaged", 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_seq": 0, 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_epoch": 0, 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_snaps": [], 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps": "[]", 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_bytes": 0, 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_objects": 0, 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "tiers": [], 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "tier_of": -1, 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "read_tier": -1, 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "write_tier": -1, 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_mode": "none", 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_bytes": 0, 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_objects": 0, 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T07:20:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_flush_age": 0, 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_evict_age": 0, 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profile": "", 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_params": { 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "none" 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_period": 0, 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_count": 0, 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "use_gmt_hitset": true, 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "min_read_recency_for_promote": 0, 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "min_write_recency_for_promote": 0, 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T07:20:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_search_last_n": 0, 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "grade_table": [], 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "stripe_width": 0, 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "expected_num_objects": 0, 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "fast_read": false, 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "options": { 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_max": 32, 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_min": 1 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "application_metadata": { 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": {} 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "read_balance": { 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "score_type": "Fair distribution", 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "score_acting": 6.9000000953674316, 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "score_stable": 6.9000000953674316, 2024-09-12T07:20:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "optimal_score": 0.28999999165534973, 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_acting": 2, 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_stable": 2, 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity_weighted": 0.75648117065429688, 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity_weighted": 0.75999999046325684 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 2, 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": "rbd", 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "create_time": "2024-09-12T07:19:28.593632+0000", 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 8193, 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type": 1, 2024-09-12T07:20:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 2, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": 1, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_rule": 0, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_count": 0, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_target": 0, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "is_stretch_pool": false, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "object_hash": 2, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscale_mode": "off", 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": 8, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num": 8, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num_target": 8, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_target": 8, 2024-09-12T07:20:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_pending": 8, 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "last_pg_merge_meta": { 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "source_pgid": "0.0", 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "ready_epoch": 0, 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_started": 0, 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_clean": 0, 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "source_version": "0'0", 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "target_version": "0'0" 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "last_change": "22", 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend": "0", 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "auid": 0, 2024-09-12T07:20:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_mode": "selfmanaged", 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_seq": 2, 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_epoch": 22, 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_snaps": [], 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps": "[]", 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_bytes": 0, 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_objects": 0, 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tiers": [], 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tier_of": -1, 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "read_tier": -1, 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "write_tier": -1, 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_mode": "none", 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_bytes": 0, 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_objects": 0, 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T07:20:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_flush_age": 0, 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_evict_age": 0, 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profile": "", 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_params": { 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "none" 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_period": 0, 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_count": 0, 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "use_gmt_hitset": true, 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "min_read_recency_for_promote": 0, 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "min_write_recency_for_promote": 0, 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T07:20:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_search_last_n": 0, 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "grade_table": [], 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "stripe_width": 0, 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "expected_num_objects": 0, 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "fast_read": false, 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "options": {}, 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "application_metadata": { 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd": {} 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "read_balance": { 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "score_type": "Fair distribution", 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "score_acting": 2.630000114440918, 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "score_stable": 2.630000114440918, 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "optimal_score": 1, 2024-09-12T07:20:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_acting": 2.630000114440918, 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_stable": 2.630000114440918, 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity_weighted": 0.75648117065429688, 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity_weighted": 0.75999999046325684 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 3, 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": "pool_client.0", 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "create_time": "2024-09-12T07:19:40.555421+0000", 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 8193, 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "type": 1, 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 2, 2024-09-12T07:20:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": 1, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_rule": 0, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_count": 0, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_target": 0, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "is_stretch_pool": false, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "object_hash": 2, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscale_mode": "off", 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": 32, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num": 32, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num_target": 32, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_target": 32, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_pending": 32, 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "last_pg_merge_meta": { 2024-09-12T07:20:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "source_pgid": "0.0", 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "ready_epoch": 0, 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_started": 0, 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_clean": 0, 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "source_version": "0'0", 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "target_version": "0'0" 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "last_change": "80", 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend": "0", 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "auid": 0, 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_mode": "selfmanaged", 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_seq": 34, 2024-09-12T07:20:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_epoch": 80, 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_snaps": [], 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps": "[]", 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_bytes": 0, 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_objects": 0, 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tiers": [], 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tier_of": -1, 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "read_tier": -1, 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "write_tier": -1, 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_mode": "none", 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_bytes": 0, 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_objects": 0, 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T07:20:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_flush_age": 0, 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_evict_age": 0, 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profile": "", 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_params": { 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "none" 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_period": 0, 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_count": 0, 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "use_gmt_hitset": true, 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "min_read_recency_for_promote": 0, 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "min_write_recency_for_promote": 0, 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_search_last_n": 0, 2024-09-12T07:20:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "grade_table": [], 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "stripe_width": 0, 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "expected_num_objects": 0, 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "fast_read": false, 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "options": {}, 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "application_metadata": { 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd": {} 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "read_balance": { 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "score_type": "Fair distribution", 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "score_acting": 5.309999942779541, 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "score_stable": 1.75, 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "optimal_score": 1.1399999856948853, 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_acting": 6.0500001907348633, 2024-09-12T07:20:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_stable": 2, 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity_weighted": 0.75648117065429688, 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity_weighted": 0.75999999046325684, 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "error_message": "pool 3 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate" 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "osds": [ 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 0, 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "710c25c3-92d3-4b51-a7fc-670879faa69c", 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 75, 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6816", 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6817", 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6818", 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6819", 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6822", 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6823", 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6820", 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6821", 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:20:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6817/4081287959", 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6819/4081287959", 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6823/4081287959", 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6821/4081287959", 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 1, 2024-09-12T07:20:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "04479191-c52e-4b60-aa0d-3e911f869062", 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 49, 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6824", 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6825", 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6826", 2024-09-12T07:20:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6827", 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6830", 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6831", 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6828", 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6829", 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6825/3980488213", 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6827/3980488213", 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3980488213", 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3980488213", 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 2, 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "ba673e51-96aa-424b-a4df-26432d633ebe", 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 46, 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6800", 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6801", 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6802", 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6803", 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.808 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6806", 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6807", 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.809 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6804", 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6805", 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6801/656802655", 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6803/656802655", 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6807/656802655", 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6805/656802655", 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:41.810 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 3, 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "c516e59c-834c-4fb0-9ff8-bcc80b21d28a", 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 46, 2024-09-12T07:20:41.811 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6808", 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6809", 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6810", 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6811", 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6814", 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6815", 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:41.814 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6812", 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6813", 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6809/3848260327", 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6811/3848260327", 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6815/3848260327", 2024-09-12T07:20:41.815 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6813/3848260327", 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 4, 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "450a70cd-ebb2-48ed-a766-184f58e7032e", 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 0.051849365234375, 2024-09-12T07:20:41.816 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 0, 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6816", 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6817", 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6818", 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6819", 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6822", 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.819 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6823", 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6820", 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6821", 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:20:41.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6817/3464256744", 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6819/3464256744", 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6823/3464256744", 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6821/3464256744", 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 5, 2024-09-12T07:20:41.821 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "a4efa333-bf57-45e7-a281-1d3bc9d2e82b", 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 46, 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.822 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6824", 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6825", 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.823 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6826", 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6827", 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6830", 2024-09-12T07:20:41.824 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6831", 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6828", 2024-09-12T07:20:41.825 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6829", 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6825/1789414693", 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6827/1789414693", 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6831/1789414693", 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6829/1789414693", 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:41.826 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 6, 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "bcaad352-f9ee-426c-bcbf-39c8d031aae0", 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 0, 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 33, 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:41.827 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6800", 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6801", 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6802", 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6804", 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6810", 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6813", 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.830 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6806", 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6808", 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6801/2576774646", 2024-09-12T07:20:41.831 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6804/2576774646", 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6813/2576774646", 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6808/2576774646", 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 7, 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "97e5ef36-58a4-4667-9f9e-b8ced5904d69", 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:20:41.832 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 75, 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6803", 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6805", 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6807", 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6809", 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6814", 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.835 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6815", 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6811", 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6812", 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6805/368918691", 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6809/368918691", 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6815/368918691", 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6812/368918691", 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:41.837 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_xinfo": [ 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 0, 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:18.862785+0000", 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 1, 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:41.838 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.059718+0000", 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 2, 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:18.575165+0000", 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:41.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 3, 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:18.847666+0000", 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 4, 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:41.840 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.471904+0000", 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 5, 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.061712+0000", 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 6, 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.541383+0000", 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 7, 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:20:41.842 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.072272+0000", 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_upmap": [], 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_upmap_items": [], 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_upmap_primaries": [], 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_temp": [], 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_temp": [], 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklist": {}, 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "range_blocklist": {}, 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profiles": { 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "default": { 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-failure-domain": "osd", 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "k": "2", 2024-09-12T07:20:41.843 INFO:teuthology.orchestra.run.smithi046.stdout: "m": "1", 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "plugin": "jerasure", 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "technique": "reed_sol_van" 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps_queue": [], 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "new_removed_snaps": [], 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "new_purged_snaps": [], 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_node_flags": {}, 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "device_class_flags": {}, 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_mode": { 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_mode_enabled": false, 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_bucket_count": 0, 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "degraded_stretch_mode": 0, 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "recovering_stretch_mode": 0, 2024-09-12T07:20:41.844 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_mode_bucket": 0 2024-09-12T07:20:41.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.845 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:41.846 INFO:tasks.thrashosds.thrasher:j is {'epoch': 80, 'fsid': '8412ed58-ac5e-4549-b523-1a58cf9f9429', 'created': '2024-09-12T07:19:10.328057+0000', 'modified': '2024-09-12T07:20:41.338641+0000', 'last_up_change': '2024-09-12T07:19:22.177473+0000', 'last_in_change': '2024-09-12T07:20:06.162815+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T07:19:23.507151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7564811706542969, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.7599999904632568}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T07:19:28.593632+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.7564811706542969, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.7599999904632568}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T07:19:40.555421+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 34, 'snap_epoch': 80, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5.309999942779541, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 6.050000190734863, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7564811706542969, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.7599999904632568, 'error_message': 'pool 3 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate'}}], 'osds': [{'osd': 0, 'uuid': '710c25c3-92d3-4b51-a7fc-670879faa69c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6817', 'nonce': 4081287959}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6818', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 4081287959}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6822', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6823', 'nonce': 4081287959}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6821', 'nonce': 4081287959}]}, 'public_addr': '172.21.15.46:6817/4081287959', 'cluster_addr': '172.21.15.46:6819/4081287959', 'heartbeat_back_addr': '172.21.15.46:6823/4081287959', 'heartbeat_front_addr': '172.21.15.46:6821/4081287959', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '04479191-c52e-4b60-aa0d-3e911f869062', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6824', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 3980488213}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6827', 'nonce': 3980488213}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3980488213}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6828', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3980488213}]}, 'public_addr': '172.21.15.46:6825/3980488213', 'cluster_addr': '172.21.15.46:6827/3980488213', 'heartbeat_back_addr': '172.21.15.46:6831/3980488213', 'heartbeat_front_addr': '172.21.15.46:6829/3980488213', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'ba673e51-96aa-424b-a4df-26432d633ebe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6801', 'nonce': 656802655}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6802', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 656802655}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6806', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 656802655}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6805', 'nonce': 656802655}]}, 'public_addr': '172.21.15.46:6801/656802655', 'cluster_addr': '172.21.15.46:6803/656802655', 'heartbeat_back_addr': '172.21.15.46:6807/656802655', 'heartbeat_front_addr': '172.21.15.46:6805/656802655', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c516e59c-834c-4fb0-9ff8-bcc80b21d28a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6809', 'nonce': 3848260327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6810', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 3848260327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6814', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 3848260327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6813', 'nonce': 3848260327}]}, 'public_addr': '172.21.15.46:6809/3848260327', 'cluster_addr': '172.21.15.46:6811/3848260327', 'heartbeat_back_addr': '172.21.15.46:6815/3848260327', 'heartbeat_front_addr': '172.21.15.46:6813/3848260327', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '450a70cd-ebb2-48ed-a766-184f58e7032e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.051849365234375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6816', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6817', 'nonce': 3464256744}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6818', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6819', 'nonce': 3464256744}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6822', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6823', 'nonce': 3464256744}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6820', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6821', 'nonce': 3464256744}]}, 'public_addr': '172.21.15.64:6817/3464256744', 'cluster_addr': '172.21.15.64:6819/3464256744', 'heartbeat_back_addr': '172.21.15.64:6823/3464256744', 'heartbeat_front_addr': '172.21.15.64:6821/3464256744', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a4efa333-bf57-45e7-a281-1d3bc9d2e82b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6824', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6825', 'nonce': 1789414693}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6826', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6827', 'nonce': 1789414693}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6830', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6831', 'nonce': 1789414693}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6828', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6829', 'nonce': 1789414693}]}, 'public_addr': '172.21.15.64:6825/1789414693', 'cluster_addr': '172.21.15.64:6827/1789414693', 'heartbeat_back_addr': '172.21.15.64:6831/1789414693', 'heartbeat_front_addr': '172.21.15.64:6829/1789414693', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'bcaad352-f9ee-426c-bcbf-39c8d031aae0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6800', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6801', 'nonce': 2576774646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6802', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6804', 'nonce': 2576774646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6810', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6813', 'nonce': 2576774646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6806', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6808', 'nonce': 2576774646}]}, 'public_addr': '172.21.15.64:6801/2576774646', 'cluster_addr': '172.21.15.64:6804/2576774646', 'heartbeat_back_addr': '172.21.15.64:6813/2576774646', 'heartbeat_front_addr': '172.21.15.64:6808/2576774646', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '97e5ef36-58a4-4667-9f9e-b8ced5904d69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6803', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6805', 'nonce': 368918691}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6807', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6809', 'nonce': 368918691}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6814', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6815', 'nonce': 368918691}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6811', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6812', 'nonce': 368918691}]}, 'public_addr': '172.21.15.64:6805/368918691', 'cluster_addr': '172.21.15.64:6809/368918691', 'heartbeat_back_addr': '172.21.15.64:6815/368918691', 'heartbeat_front_addr': '172.21.15.64:6812/368918691', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:18.862785+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.059718+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:18.575165+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:18.847666+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.471904+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.061712+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.541383+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.072272+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T07:20:41.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:20:41.847 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:41.847+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:41.864 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:41.864 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:41.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:41.864 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:41.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.864 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82749 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 429056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.220738157, 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055052166, 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82749 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0800000000000001e-07 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644459+0000", 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1769999999999999e-06 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644462+0000", 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8779999999999998e-06 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644465+0000", 2024-09-12T07:20:41.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6509999999999999e-06 2024-09-12T07:20:41.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699279+0000", 2024-09-12T07:20:41.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054813832999999999 2024-09-12T07:20:41.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699359+0000", 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0150999999999999e-05 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699510+0000", 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015026800000000001 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.869 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82750 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 432128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.220725272999999, 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055234722, 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82750 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644471+0000", 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.17e-07 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644472+0000", 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.19e-06 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644482+0000", 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0332e-05 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644489+0000", 2024-09-12T07:20:41.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7039999999999997e-06 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699549+0000", 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055060588000000001 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699592+0000", 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2707000000000003e-05 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699705+0000", 2024-09-12T07:20:41.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112984 2024-09-12T07:20:41.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.874 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.876 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82751 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 436224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.876 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:41.876 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.220697919000003, 2024-09-12T07:20:41.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055415691000000003, 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82751 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644846+0000", 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034807600000000001 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644850+0000", 2024-09-12T07:20:41.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1649999999999999e-06 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644860+0000", 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6949999999999995e-06 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644866+0000", 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3640000000000004e-06 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699735+0000", 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054868888999999997 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699776+0000", 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0722000000000001e-05 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699913+0000", 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013778 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.880 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82752 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 439296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.220320176000001, 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226917, 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.881 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82752 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644878+0000", 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7850000000000001e-06 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644882+0000", 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9990000000000002e-06 2024-09-12T07:20:41.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644885+0000", 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.965e-06 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699952+0000", 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067243000000002 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699989+0000", 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7450999999999998e-05 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700102+0000", 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113124 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82753 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 443392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:41.885 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.220307519999999, 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055422102000000001, 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82753 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645016+0000", 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000127847 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645024+0000", 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7419999999999997e-06 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645030+0000", 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7229999999999998e-06 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645034+0000", 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9380000000000002e-06 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700132+0000", 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055097699 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700171+0000", 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9093e-05 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700310+0000", 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013905999999999999 2024-09-12T07:20:41.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82754 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 446464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.220151612999999, 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055424703999999998, 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82754 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645318+0000", 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027397499999999998 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645322+0000", 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.766e-06 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645331+0000", 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2140000000000002e-06 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645338+0000", 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0929999999999997e-06 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700330+0000", 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054991384999999997 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700369+0000", 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700469+0000", 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9913000000000004e-05 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82755 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 449536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.219846623000002, 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055388918000000002, 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82755 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645500+0000", 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015050999999999999 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645502+0000", 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-12T07:20:41.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645509+0000", 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3969999999999999e-06 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645513+0000", 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9230000000000004e-06 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700507+0000", 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054994692999999997 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700583+0000", 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5656999999999997e-05 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700738+0000", 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154869 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.898 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82756 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 453632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.219675735000003, 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055451746000000003, 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.899 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82756 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645763+0000", 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000243105 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645766+0000", 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0750000000000002e-06 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645774+0000", 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9370000000000004e-06 2024-09-12T07:20:41.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645783+0000", 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5159999999999994e-06 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700780+0000", 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054997329999999997 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700835+0000", 2024-09-12T07:20:41.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4789e-05 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700972+0000", 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013699400000000001 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82757 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.219401367000003, 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055382590000000002, 2024-09-12T07:20:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82757 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645926+0000", 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131979 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645929+0000", 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0199999999999999e-06 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645935+0000", 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645939+0000", 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.924e-06 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701007+0000", 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067374000000002 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701043+0000", 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6325999999999997e-05 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701177+0000", 2024-09-12T07:20:41.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013387300000000001 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82758 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 460800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.219243974000001, 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055417890999999997, 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82758 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646164+0000", 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000212189 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646167+0000", 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646173+0000", 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4139999999999999e-06 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646177+0000", 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701208+0000", 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055031322000000001 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701254+0000", 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5701999999999997e-05 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701370+0000", 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115456 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82759 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 463872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:41.912 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.219011412999997, 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055376838999999997, 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82759 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:41.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646368+0000", 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183399 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646372+0000", 2024-09-12T07:20:41.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9190000000000001e-06 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646380+0000", 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7819999999999999e-06 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646384+0000", 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701407+0000", 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055022763000000002 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701455+0000", 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7660000000000001e-05 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701561+0000", 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010642899999999999 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.916 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82760 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.218803864000002, 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055355572999999998, 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.917 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82760 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646631+0000", 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000238798 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646634+0000", 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.016e-06 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646640+0000", 2024-09-12T07:20:41.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2750000000000001e-06 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646644+0000", 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7859999999999998e-06 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701599+0000", 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054955693999999999 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701638+0000", 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8319999999999999e-05 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701747+0000", 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109684 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82761 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.218545235000001, 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055296990999999997, 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82761 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646756+0000", 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105899 2024-09-12T07:20:41.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646759+0000", 2024-09-12T07:20:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8380000000000002e-06 2024-09-12T07:20:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646765+0000", 2024-09-12T07:20:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:20:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646769+0000", 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7900000000000001e-06 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701785+0000", 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055015888999999998 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701832+0000", 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6514000000000002e-05 2024-09-12T07:20:41.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701947+0000", 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115833 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82762 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 474112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.218419668000003, 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055346863000000003, 2024-09-12T07:20:41.926 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82762 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647063+0000", 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000287115 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647066+0000", 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0189999999999998e-06 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647075+0000", 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7099999999999996e-06 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647081+0000", 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6790000000000002e-06 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701977+0000", 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054896665999999997 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702015+0000", 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.79e-05 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702123+0000", 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107774 2024-09-12T07:20:41.930 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82763 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.218104750999998, 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226509999999999, 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82763 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647236+0000", 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000144597 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647239+0000", 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1329999999999999e-06 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647245+0000", 2024-09-12T07:20:41.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647249+0000", 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2270000000000004e-06 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702160+0000", 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054910549000000003 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702205+0000", 2024-09-12T07:20:41.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5575000000000003e-05 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702317+0000", 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011194899999999999 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82764 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 481280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:41.935 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.217939000000001, 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055229250000000001, 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82764 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647471+0000", 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000214625 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647474+0000", 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8509999999999999e-06 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647480+0000", 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.28e-06 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647484+0000", 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.771e-06 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702346+0000", 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054861551000000001 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702382+0000", 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.65e-05 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702486+0000", 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010367200000000001 2024-09-12T07:20:41.939 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82765 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 485376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.217703452999999, 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055184729000000002, 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82765 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647681+0000", 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188965 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647693+0000", 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.225e-05 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647700+0000", 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9929999999999998e-06 2024-09-12T07:20:41.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647704+0000", 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9469999999999996e-06 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702522+0000", 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054817346000000003 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702566+0000", 2024-09-12T07:20:41.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4672999999999997e-05 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702677+0000", 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110555 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82766 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 488448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.217484214999999, 2024-09-12T07:20:41.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055132717999999997, 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82766 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647959+0000", 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024739599999999999 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647964+0000", 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9409999999999998e-06 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647975+0000", 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1404e-05 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647983+0000", 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2969999999999998e-06 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702706+0000", 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054723690999999998 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702743+0000", 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6464999999999997e-05 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702844+0000", 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101524 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.948 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82767 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 492544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.217200173999998, 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055054448999999998, 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82767 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648182+0000", 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018652299999999999 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648185+0000", 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3289999999999998e-06 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648192+0000", 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7159999999999997e-06 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648196+0000", 2024-09-12T07:20:41.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1439999999999996e-06 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702873+0000", 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054676356000000002 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702930+0000", 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7643999999999999e-05 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703050+0000", 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119737 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:148945 3.10 3:0cac0a45:::rbd_data.1160161ce0c8.000000000000009d:head [delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4313942129999999, 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89878390500000005, 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:41.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 148945 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433803+0000", 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.164e-06 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433807+0000", 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5000000000000001e-06 2024-09-12T07:20:41.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433816+0000", 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1859999999999998e-06 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433822+0000", 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5300000000000004e-06 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433886+0000", 2024-09-12T07:20:41.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4510000000000004e-05 2024-09-12T07:20:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:20:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433896+0000", 2024-09-12T07:20:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0012e-05 2024-09-12T07:20:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329359+0000", 2024-09-12T07:20:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89546257399999996 2024-09-12T07:20:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329585+0000", 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022641800000000001 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329712+0000", 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126339 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.330704+0000", 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099219900000000003 2024-09-12T07:20:41.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332513+0000", 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001808689 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332544+0000", 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1186999999999998e-05 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332585+0000", 2024-09-12T07:20:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1597000000000002e-05 2024-09-12T07:20:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:41.960 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:41.960 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:41.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:20:41.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:41.963 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:41.963+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:42.063 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:42.064+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:42.116 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:42.116 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:42.116 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:42.117 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:42.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:42.137 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:42.156 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:42.156 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:42.163 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:42.165+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:42.175 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:42.175 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:42.209 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:42.227 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:42.227 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:42.244 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:42.246 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:42.247 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:42.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:42.261 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:42.262 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:42.264 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:42.265+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:42.278 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:20:42.278 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:20:42.290 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-12T07:20:41.502720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1040143239,"num_objects":1345,"num_object_clones":331,"num_object_copies":2690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1345,"num_whiteouts":612,"num_read":175195,"num_read_kb":858724,"num_write":12013,"num_write_kb":459084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":49942512,"num_keys_recovered":48,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9671,"ondisk_log_size":9671,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":887852,"kb_used_data":592572,"kb_used_omap":12,"kb_used_meta":292339,"kb_avail":754086868,"statfs":{"total":773094113280,"available":772184952832,"internally_reserved":0,"allocated":606793728,"data_stored":709041313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":299355697},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":91015947,"num_objects":119,"num_object_clones":35,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":48,"num_read":10327,"num_read_kb":85856,"num_write":1591,"num_write_kb":45917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000645"},"pg_stats":[{"pgid":"3.1f","version":"74'354","reported_seq":2051,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.326885+0000","last_change":"2024-09-12T07:19:44.455827+0000","last_active":"2024-09-12T07:20:37.326885+0000","last_peered":"2024-09-12T07:20:37.326885+0000","last_clean":"2024-09-12T07:20:37.326885+0000","last_became_active":"2024-09-12T07:19:42.451212+0000","last_became_peered":"2024-09-12T07:19:42.451212+0000","last_unstale":"2024-09-12T07:20:37.326885+0000","last_undegraded":"2024-09-12T07:20:37.326885+0000","last_fullsized":"2024-09-12T07:20:37.326885+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:12.937860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001023672,"stat_sum":{"num_bytes":13266909,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":3204,"num_read_kb":15942,"num_write":483,"num_write_kb":5954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"75'232","reported_seq":993,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.323957+0000","last_change":"2024-09-12T07:20:07.305432+0000","last_active":"2024-09-12T07:20:37.323957+0000","last_peered":"2024-09-12T07:20:37.323957+0000","last_clean":"2024-09-12T07:20:37.323957+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:20:37.323957+0000","last_undegraded":"2024-09-12T07:20:37.323957+0000","last_fullsized":"2024-09-12T07:20:37.323957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:21.520090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081935099999999996,"stat_sum":{"num_bytes":47819601,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":17794,"num_read_kb":34165,"num_write":213,"num_write_kb":18813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"73'187","reported_seq":18323,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.323381+0000","last_change":"2024-09-12T07:19:44.464376+0000","last_active":"2024-09-12T07:20:37.323381+0000","last_peered":"2024-09-12T07:20:37.323381+0000","last_clean":"2024-09-12T07:20:37.323381+0000","last_became_active":"2024-09-12T07:19:42.451973+0000","last_became_peered":"2024-09-12T07:19:42.451973+0000","last_unstale":"2024-09-12T07:20:37.323381+0000","last_undegraded":"2024-09-12T07:20:37.323381+0000","last_fullsized":"2024-09-12T07:20:37.323381+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:02:03.517118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036949999999999,"stat_sum":{"num_bytes":30299710,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":18064,"num_read_kb":30545,"num_write":166,"num_write_kb":12283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"76'414","reported_seq":2559,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.368841+0000","last_change":"2024-09-12T07:19:44.464287+0000","last_active":"2024-09-12T07:20:37.368841+0000","last_peered":"2024-09-12T07:20:37.368841+0000","last_clean":"2024-09-12T07:20:37.368841+0000","last_became_active":"2024-09-12T07:19:42.455338+0000","last_became_peered":"2024-09-12T07:19:42.455338+0000","last_unstale":"2024-09-12T07:20:37.368841+0000","last_undegraded":"2024-09-12T07:20:37.368841+0000","last_fullsized":"2024-09-12T07:20:37.368841+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:58:43.984661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012347180000000001,"stat_sum":{"num_bytes":30848886,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":3158,"num_read_kb":14725,"num_write":575,"num_write_kb":8311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"74'283","reported_seq":17751,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.326769+0000","last_change":"2024-09-12T07:19:44.456652+0000","last_active":"2024-09-12T07:20:37.326769+0000","last_peered":"2024-09-12T07:20:37.326769+0000","last_clean":"2024-09-12T07:20:37.326769+0000","last_became_active":"2024-09-12T07:19:42.454990+0000","last_became_peered":"2024-09-12T07:19:42.454990+0000","last_unstale":"2024-09-12T07:20:37.326769+0000","last_undegraded":"2024-09-12T07:20:37.326769+0000","last_fullsized":"2024-09-12T07:20:37.326769+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:42:41.337887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049866,"stat_sum":{"num_bytes":31412648,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":18274,"num_read_kb":45982,"num_write":296,"num_write_kb":14139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"75'264","reported_seq":2727,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.382316+0000","last_change":"2024-09-12T07:19:44.459417+0000","last_active":"2024-09-12T07:20:37.382316+0000","last_peered":"2024-09-12T07:20:37.382316+0000","last_clean":"2024-09-12T07:20:37.382316+0000","last_became_active":"2024-09-12T07:19:42.454620+0000","last_became_peered":"2024-09-12T07:19:42.454620+0000","last_unstale":"2024-09-12T07:20:37.382316+0000","last_undegraded":"2024-09-12T07:20:37.382316+0000","last_fullsized":"2024-09-12T07:20:37.382316+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:44.310037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090501500000000001,"stat_sum":{"num_bytes":17783530,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":23,"num_read":3310,"num_read_kb":10719,"num_write":251,"num_write_kb":15095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"75'303","reported_seq":4907,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.342992+0000","last_change":"2024-09-12T07:20:10.196744+0000","last_active":"2024-09-12T07:20:37.342992+0000","last_peered":"2024-09-12T07:20:37.342992+0000","last_clean":"2024-09-12T07:20:37.342992+0000","last_became_active":"2024-09-12T07:20:10.195312+0000","last_became_peered":"2024-09-12T07:20:10.195312+0000","last_unstale":"2024-09-12T07:20:37.342992+0000","last_undegraded":"2024-09-12T07:20:37.342992+0000","last_fullsized":"2024-09-12T07:20:37.342992+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:32:12.637789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113156,"stat_sum":{"num_bytes":27214545,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":5266,"num_read_kb":38807,"num_write":406,"num_write_kb":12763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1101525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"74'341","reported_seq":2691,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.420601+0000","last_change":"2024-09-12T07:20:07.446518+0000","last_active":"2024-09-12T07:20:37.420601+0000","last_peered":"2024-09-12T07:20:37.420601+0000","last_clean":"2024-09-12T07:20:37.420601+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:20:37.420601+0000","last_undegraded":"2024-09-12T07:20:37.420601+0000","last_fullsized":"2024-09-12T07:20:37.420601+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:23:26.296381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000443294,"stat_sum":{"num_bytes":48690557,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":4966,"num_read_kb":40890,"num_write":442,"num_write_kb":17677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"75'324","reported_seq":4832,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.420659+0000","last_change":"2024-09-12T07:19:44.456115+0000","last_active":"2024-09-12T07:20:37.420659+0000","last_peered":"2024-09-12T07:20:37.420659+0000","last_clean":"2024-09-12T07:20:37.420659+0000","last_became_active":"2024-09-12T07:19:42.452617+0000","last_became_peered":"2024-09-12T07:19:42.452617+0000","last_unstale":"2024-09-12T07:20:37.420659+0000","last_undegraded":"2024-09-12T07:20:37.420659+0000","last_fullsized":"2024-09-12T07:20:37.420659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:38:21.596455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077251,"stat_sum":{"num_bytes":33836508,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":16,"num_read":5817,"num_read_kb":15177,"num_write":321,"num_write_kb":20608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"75'142","reported_seq":721,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.374116+0000","last_change":"2024-09-12T07:19:44.463507+0000","last_active":"2024-09-12T07:20:37.374116+0000","last_peered":"2024-09-12T07:20:37.374116+0000","last_clean":"2024-09-12T07:20:37.374116+0000","last_became_active":"2024-09-12T07:19:42.455210+0000","last_became_peered":"2024-09-12T07:19:42.455210+0000","last_unstale":"2024-09-12T07:20:37.374116+0000","last_undegraded":"2024-09-12T07:20:37.374116+0000","last_fullsized":"2024-09-12T07:20:37.374116+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:32.801388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049753700000000004,"stat_sum":{"num_bytes":17664209,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":23,"num_read":486,"num_read_kb":6829,"num_write":150,"num_write_kb":6747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"74'154","reported_seq":5951,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.325583+0000","last_change":"2024-09-12T07:20:07.261139+0000","last_active":"2024-09-12T07:20:37.325583+0000","last_peered":"2024-09-12T07:20:37.325583+0000","last_clean":"2024-09-12T07:20:37.325583+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:20:37.325583+0000","last_undegraded":"2024-09-12T07:20:37.325583+0000","last_fullsized":"2024-09-12T07:20:37.325583+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:03.937117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012225490000000001,"stat_sum":{"num_bytes":21134923,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":5930,"num_read_kb":41383,"num_write":135,"num_write_kb":7768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"75'226","reported_seq":4678,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.325593+0000","last_change":"2024-09-12T07:19:44.481796+0000","last_active":"2024-09-12T07:20:37.325593+0000","last_peered":"2024-09-12T07:20:37.325593+0000","last_clean":"2024-09-12T07:20:37.325593+0000","last_became_active":"2024-09-12T07:19:42.452277+0000","last_became_peered":"2024-09-12T07:19:42.452277+0000","last_unstale":"2024-09-12T07:20:37.325593+0000","last_undegraded":"2024-09-12T07:20:37.325593+0000","last_fullsized":"2024-09-12T07:20:37.325593+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:10.916310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012485160000000001,"stat_sum":{"num_bytes":40023861,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":4828,"num_read_kb":56928,"num_write":226,"num_write_kb":28024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.329831+0000","last_change":"2024-09-12T07:20:37.329418+0000","last_active":"2024-09-12T07:20:37.329831+0000","last_peered":"2024-09-12T07:20:37.329831+0000","last_clean":"2024-09-12T07:20:37.329831+0000","last_became_active":"2024-09-12T07:20:37.329145+0000","last_became_peered":"2024-09-12T07:20:37.329145+0000","last_unstale":"2024-09-12T07:20:37.329831+0000","last_undegraded":"2024-09-12T07:20:37.329831+0000","last_fullsized":"2024-09-12T07:20:37.329831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:58.555569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"73'330","reported_seq":1506,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.327023+0000","last_change":"2024-09-12T07:19:44.455257+0000","last_active":"2024-09-12T07:20:37.327023+0000","last_peered":"2024-09-12T07:20:37.327023+0000","last_clean":"2024-09-12T07:20:37.327023+0000","last_became_active":"2024-09-12T07:19:42.449730+0000","last_became_peered":"2024-09-12T07:19:42.449730+0000","last_unstale":"2024-09-12T07:20:37.327023+0000","last_undegraded":"2024-09-12T07:20:37.327023+0000","last_fullsized":"2024-09-12T07:20:37.327023+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:22.673649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059259600000000003,"stat_sum":{"num_bytes":20649853,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":2144,"num_read_kb":9775,"num_write":409,"num_write_kb":8560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.343109+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:20:37.343109+0000","last_peered":"2024-09-12T07:20:37.343109+0000","last_clean":"2024-09-12T07:20:37.343109+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:20:37.343109+0000","last_undegraded":"2024-09-12T07:20:37.343109+0000","last_fullsized":"2024-09-12T07:20:37.343109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"74'311","reported_seq":1249,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.421061+0000","last_change":"2024-09-12T07:20:07.211023+0000","last_active":"2024-09-12T07:20:37.421061+0000","last_peered":"2024-09-12T07:20:37.421061+0000","last_clean":"2024-09-12T07:20:37.421061+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:20:37.421061+0000","last_undegraded":"2024-09-12T07:20:37.421061+0000","last_fullsized":"2024-09-12T07:20:37.421061+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:19:11.706234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087343100000000003,"stat_sum":{"num_bytes":25250526,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":1680,"num_read_kb":12999,"num_write":394,"num_write_kb":5388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.325585+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:20:37.325585+0000","last_peered":"2024-09-12T07:20:37.325585+0000","last_clean":"2024-09-12T07:20:37.325585+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:20:37.325585+0000","last_undegraded":"2024-09-12T07:20:37.325585+0000","last_fullsized":"2024-09-12T07:20:37.325585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'297","reported_seq":1357,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.422389+0000","last_change":"2024-09-12T07:19:44.458925+0000","last_active":"2024-09-12T07:20:37.422389+0000","last_peered":"2024-09-12T07:20:37.422389+0000","last_clean":"2024-09-12T07:20:37.422389+0000","last_became_active":"2024-09-12T07:19:42.452704+0000","last_became_peered":"2024-09-12T07:19:42.452704+0000","last_unstale":"2024-09-12T07:20:37.422389+0000","last_undegraded":"2024-09-12T07:20:37.422389+0000","last_fullsized":"2024-09-12T07:20:37.422389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:15.384077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045577200000000001,"stat_sum":{"num_bytes":31139649,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":18,"num_read":1875,"num_read_kb":7225,"num_write":400,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.343202+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:20:37.343202+0000","last_peered":"2024-09-12T07:20:37.343202+0000","last_clean":"2024-09-12T07:20:37.343202+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:20:37.343202+0000","last_undegraded":"2024-09-12T07:20:37.343202+0000","last_fullsized":"2024-09-12T07:20:37.343202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'238","reported_seq":4600,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.323409+0000","last_change":"2024-09-12T07:20:07.289858+0000","last_active":"2024-09-12T07:20:37.323409+0000","last_peered":"2024-09-12T07:20:37.323409+0000","last_clean":"2024-09-12T07:20:37.323409+0000","last_became_active":"2024-09-12T07:20:07.183819+0000","last_became_peered":"2024-09-12T07:20:07.183819+0000","last_unstale":"2024-09-12T07:20:37.323409+0000","last_undegraded":"2024-09-12T07:20:37.323409+0000","last_fullsized":"2024-09-12T07:20:37.323409+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:53:02.543942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015134619999999999,"stat_sum":{"num_bytes":50378098,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":5039,"num_read_kb":50196,"num_write":317,"num_write_kb":20520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":3070354,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'177","reported_seq":1897,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.327038+0000","last_change":"2024-09-12T07:19:44.456465+0000","last_active":"2024-09-12T07:20:37.327038+0000","last_peered":"2024-09-12T07:20:37.327038+0000","last_clean":"2024-09-12T07:20:37.327038+0000","last_became_active":"2024-09-12T07:19:42.454103+0000","last_became_peered":"2024-09-12T07:19:42.454103+0000","last_unstale":"2024-09-12T07:20:37.327038+0000","last_undegraded":"2024-09-12T07:20:37.327038+0000","last_fullsized":"2024-09-12T07:20:37.327038+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:04:37.059623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010980880000000001,"stat_sum":{"num_bytes":20996207,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":1629,"num_read_kb":5759,"num_write":160,"num_write_kb":8185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":130,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.323225+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:20:37.323225+0000","last_peered":"2024-09-12T07:20:37.323225+0000","last_clean":"2024-09-12T07:20:37.323225+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:20:37.323225+0000","last_undegraded":"2024-09-12T07:20:37.323225+0000","last_fullsized":"2024-09-12T07:20:37.323225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"74'520","reported_seq":2265,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.342983+0000","last_change":"2024-09-12T07:19:44.458868+0000","last_active":"2024-09-12T07:20:37.342983+0000","last_peered":"2024-09-12T07:20:37.342983+0000","last_clean":"2024-09-12T07:20:37.342983+0000","last_became_active":"2024-09-12T07:19:42.451807+0000","last_became_peered":"2024-09-12T07:19:42.451807+0000","last_unstale":"2024-09-12T07:20:37.342983+0000","last_undegraded":"2024-09-12T07:20:37.342983+0000","last_fullsized":"2024-09-12T07:20:37.342983+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:25:27.932545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048363399999999998,"stat_sum":{"num_bytes":29481718,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":4234,"num_read_kb":9692,"num_write":774,"num_write_kb":9863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":140,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.336319+0000","last_change":"2024-09-12T07:20:07.183543+0000","last_active":"2024-09-12T07:20:37.336319+0000","last_peered":"2024-09-12T07:20:37.336319+0000","last_clean":"2024-09-12T07:20:37.336319+0000","last_became_active":"2024-09-12T07:20:07.183213+0000","last_became_peered":"2024-09-12T07:20:07.183213+0000","last_unstale":"2024-09-12T07:20:37.336319+0000","last_undegraded":"2024-09-12T07:20:37.336319+0000","last_fullsized":"2024-09-12T07:20:37.336319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.634459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"75'177","reported_seq":2899,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.326688+0000","last_change":"2024-09-12T07:19:44.455560+0000","last_active":"2024-09-12T07:20:37.326688+0000","last_peered":"2024-09-12T07:20:37.326688+0000","last_clean":"2024-09-12T07:20:37.326688+0000","last_became_active":"2024-09-12T07:19:42.451970+0000","last_became_peered":"2024-09-12T07:19:42.451970+0000","last_unstale":"2024-09-12T07:20:37.326688+0000","last_undegraded":"2024-09-12T07:20:37.326688+0000","last_fullsized":"2024-09-12T07:20:37.326688+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:04.772264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083352100000000004,"stat_sum":{"num_bytes":43961201,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":3554,"num_read_kb":33706,"num_write":188,"num_write_kb":23259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.325457+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:20:37.325457+0000","last_peered":"2024-09-12T07:20:37.325457+0000","last_clean":"2024-09-12T07:20:37.325457+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:20:37.325457+0000","last_undegraded":"2024-09-12T07:20:37.325457+0000","last_fullsized":"2024-09-12T07:20:37.325457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'707","reported_seq":3209,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.323794+0000","last_change":"2024-09-12T07:20:07.467030+0000","last_active":"2024-09-12T07:20:37.323794+0000","last_peered":"2024-09-12T07:20:37.323794+0000","last_clean":"2024-09-12T07:20:37.323794+0000","last_became_active":"2024-09-12T07:20:07.180577+0000","last_became_peered":"2024-09-12T07:20:07.180577+0000","last_unstale":"2024-09-12T07:20:37.323794+0000","last_undegraded":"2024-09-12T07:20:37.323794+0000","last_fullsized":"2024-09-12T07:20:37.323794+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:11:26.147779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089638599999999999,"stat_sum":{"num_bytes":36186712,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":24,"num_read":5422,"num_read_kb":37393,"num_write":1127,"num_write_kb":14737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":261251,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":129,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.323224+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:20:37.323224+0000","last_peered":"2024-09-12T07:20:37.323224+0000","last_clean":"2024-09-12T07:20:37.323224+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:20:37.323224+0000","last_undegraded":"2024-09-12T07:20:37.323224+0000","last_fullsized":"2024-09-12T07:20:37.323224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":219,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.325350+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:20:37.325350+0000","last_peered":"2024-09-12T07:20:37.325350+0000","last_clean":"2024-09-12T07:20:37.325350+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:20:37.325350+0000","last_undegraded":"2024-09-12T07:20:37.325350+0000","last_fullsized":"2024-09-12T07:20:37.325350+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"75'794","reported_seq":1664,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.353486+0000","last_change":"2024-09-12T07:20:07.428482+0000","last_active":"2024-09-12T07:20:37.353486+0000","last_peered":"2024-09-12T07:20:37.353486+0000","last_clean":"2024-09-12T07:20:37.353486+0000","last_became_active":"2024-09-12T07:20:07.180763+0000","last_became_peered":"2024-09-12T07:20:07.180763+0000","last_unstale":"2024-09-12T07:20:37.353486+0000","last_undegraded":"2024-09-12T07:20:37.353486+0000","last_fullsized":"2024-09-12T07:20:37.353486+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:58.492449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047491800000000001,"stat_sum":{"num_bytes":35566196,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":21,"num_read":4453,"num_read_kb":20399,"num_write":1308,"num_write_kb":22218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":8633990,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"76'357","reported_seq":3960,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.420245+0000","last_change":"2024-09-12T07:19:44.469508+0000","last_active":"2024-09-12T07:20:37.420245+0000","last_peered":"2024-09-12T07:20:37.420245+0000","last_clean":"2024-09-12T07:20:37.420245+0000","last_became_active":"2024-09-12T07:19:42.450230+0000","last_became_peered":"2024-09-12T07:19:42.450230+0000","last_unstale":"2024-09-12T07:20:37.420245+0000","last_undegraded":"2024-09-12T07:20:37.420245+0000","last_fullsized":"2024-09-12T07:20:37.420245+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:15.610202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076460899999999997,"stat_sum":{"num_bytes":35040089,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":23,"num_read":5804,"num_read_kb":28907,"num_write":365,"num_write_kb":10343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"75'213","reported_seq":3281,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.328200+0000","last_change":"2024-09-12T07:19:44.480981+0000","last_active":"2024-09-12T07:20:37.328200+0000","last_peered":"2024-09-12T07:20:37.328200+0000","last_clean":"2024-09-12T07:20:37.328200+0000","last_became_active":"2024-09-12T07:19:42.452637+0000","last_became_peered":"2024-09-12T07:19:42.452637+0000","last_unstale":"2024-09-12T07:20:37.328200+0000","last_undegraded":"2024-09-12T07:20:37.328200+0000","last_fullsized":"2024-09-12T07:20:37.328200+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:55:02.267604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000680892,"stat_sum":{"num_bytes":35360399,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":25,"num_read":3094,"num_read_kb":31085,"num_write":229,"num_write_kb":17250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"75'230","reported_seq":1442,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.323875+0000","last_change":"2024-09-12T07:20:07.379907+0000","last_active":"2024-09-12T07:20:37.323875+0000","last_peered":"2024-09-12T07:20:37.323875+0000","last_clean":"2024-09-12T07:20:37.323875+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:20:37.323875+0000","last_undegraded":"2024-09-12T07:20:37.323875+0000","last_fullsized":"2024-09-12T07:20:37.323875+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:37.810437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054644100000000005,"stat_sum":{"num_bytes":32795886,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":2125,"num_read_kb":14694,"num_write":230,"num_write_kb":15154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"76'299","reported_seq":8649,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.421010+0000","last_change":"2024-09-12T07:20:07.249347+0000","last_active":"2024-09-12T07:20:37.421010+0000","last_peered":"2024-09-12T07:20:37.421010+0000","last_clean":"2024-09-12T07:20:37.421010+0000","last_became_active":"2024-09-12T07:20:07.183065+0000","last_became_peered":"2024-09-12T07:20:07.183065+0000","last_unstale":"2024-09-12T07:20:37.421010+0000","last_undegraded":"2024-09-12T07:20:37.421010+0000","last_fullsized":"2024-09-12T07:20:37.421010+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:48.650291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050446300000000003,"stat_sum":{"num_bytes":46536311,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":9663,"num_read_kb":47023,"num_write":270,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":8578058,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"73'179","reported_seq":1752,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.325476+0000","last_change":"2024-09-12T07:19:44.482470+0000","last_active":"2024-09-12T07:20:37.325476+0000","last_peered":"2024-09-12T07:20:37.325476+0000","last_clean":"2024-09-12T07:20:37.325476+0000","last_became_active":"2024-09-12T07:19:42.450577+0000","last_became_peered":"2024-09-12T07:19:42.450577+0000","last_unstale":"2024-09-12T07:20:37.325476+0000","last_undegraded":"2024-09-12T07:20:37.325476+0000","last_fullsized":"2024-09-12T07:20:37.325476+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:23.473857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317836,"stat_sum":{"num_bytes":34488353,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":1816,"num_read_kb":19339,"num_write":194,"num_write_kb":8525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"75'345","reported_seq":3907,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.325483+0000","last_change":"2024-09-12T07:19:44.481850+0000","last_active":"2024-09-12T07:20:37.325483+0000","last_peered":"2024-09-12T07:20:37.325483+0000","last_clean":"2024-09-12T07:20:37.325483+0000","last_became_active":"2024-09-12T07:19:42.452236+0000","last_became_peered":"2024-09-12T07:19:42.452236+0000","last_unstale":"2024-09-12T07:20:37.325483+0000","last_undegraded":"2024-09-12T07:20:37.325483+0000","last_fullsized":"2024-09-12T07:20:37.325483+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:44.673985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012939710000000001,"stat_sum":{"num_bytes":24966706,"num_objects":50,"num_object_clones":16,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":5321,"num_read_kb":25995,"num_write":437,"num_write_kb":11198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"75'328","reported_seq":1181,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.378375+0000","last_change":"2024-09-12T07:19:44.482179+0000","last_active":"2024-09-12T07:20:37.378375+0000","last_peered":"2024-09-12T07:20:37.378375+0000","last_clean":"2024-09-12T07:20:37.378375+0000","last_became_active":"2024-09-12T07:19:42.452155+0000","last_became_peered":"2024-09-12T07:19:42.452155+0000","last_unstale":"2024-09-12T07:20:37.378375+0000","last_undegraded":"2024-09-12T07:20:37.378375+0000","last_fullsized":"2024-09-12T07:20:37.378375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:04.847254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011884759999999999,"stat_sum":{"num_bytes":19228533,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":1950,"num_read_kb":11425,"num_write":405,"num_write_kb":14997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"76'199","reported_seq":1080,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.332545+0000","last_change":"2024-09-12T07:20:37.330497+0000","last_active":"2024-09-12T07:20:37.332545+0000","last_peered":"2024-09-12T07:20:37.332545+0000","last_clean":"2024-09-12T07:20:37.332545+0000","last_became_active":"2024-09-12T07:20:37.328982+0000","last_became_peered":"2024-09-12T07:20:37.328982+0000","last_unstale":"2024-09-12T07:20:37.332545+0000","last_undegraded":"2024-09-12T07:20:37.332545+0000","last_fullsized":"2024-09-12T07:20:37.332545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:54.919602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073032100000000003,"stat_sum":{"num_bytes":43721760,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":22,"num_read":1116,"num_read_kb":23114,"num_write":223,"num_write_kb":16374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"75'180","reported_seq":2785,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.325333+0000","last_change":"2024-09-12T07:19:44.481275+0000","last_active":"2024-09-12T07:20:37.325333+0000","last_peered":"2024-09-12T07:20:37.325333+0000","last_clean":"2024-09-12T07:20:37.325333+0000","last_became_active":"2024-09-12T07:19:42.452104+0000","last_became_peered":"2024-09-12T07:19:42.452104+0000","last_unstale":"2024-09-12T07:20:37.325333+0000","last_undegraded":"2024-09-12T07:20:37.325333+0000","last_fullsized":"2024-09-12T07:20:37.325333+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:42.138610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079083699999999996,"stat_sum":{"num_bytes":20264262,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":2978,"num_read_kb":30025,"num_write":170,"num_write_kb":15993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"76'150","reported_seq":1630,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.374545+0000","last_change":"2024-09-12T07:19:44.456086+0000","last_active":"2024-09-12T07:20:37.374545+0000","last_peered":"2024-09-12T07:20:37.374545+0000","last_clean":"2024-09-12T07:20:37.374545+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:20:37.374545+0000","last_undegraded":"2024-09-12T07:20:37.374545+0000","last_fullsized":"2024-09-12T07:20:37.374545+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:04.247341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060746,"stat_sum":{"num_bytes":31238103,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":14,"num_read":1441,"num_read_kb":18030,"num_write":189,"num_write_kb":10497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"74'341","reported_seq":18546,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-12T07:20:37.325421+0000","last_change":"2024-09-12T07:19:44.482564+0000","last_active":"2024-09-12T07:20:37.325421+0000","last_peered":"2024-09-12T07:20:37.325421+0000","last_clean":"2024-09-12T07:20:37.325421+0000","last_became_active":"2024-09-12T07:19:42.456127+0000","last_became_peered":"2024-09-12T07:19:42.456127+0000","last_unstale":"2024-09-12T07:20:37.325421+0000","last_undegraded":"2024-09-12T07:20:37.325421+0000","last_fullsized":"2024-09-12T07:20:37.325421+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:07.588163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793039999999999,"stat_sum":{"num_bytes":62306403,"num_objects":53,"num_object_clones":16,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":23,"num_read":18694,"num_read_kb":69796,"num_write":457,"num_write_kb":35556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1039552852,"num_objects":1341,"num_object_clones":331,"num_object_copies":2682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1341,"num_whiteouts":612,"num_read":175129,"num_read_kb":858669,"num_write":11910,"num_write_kb":457454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":49942512,"num_keys_recovered":48,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":598618112,"data_stored":704719969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9596,"ondisk_log_size":9596,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134296,"kb_used_data":98940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94237544,"statfs":{"total":96636764160,"available":96499245056,"internally_reserved":0,"allocated":101314560,"data_stored":117068017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63328,"kb_used_data":32132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308512,"statfs":{"total":96636764160,"available":96571916288,"internally_reserved":0,"allocated":32903168,"data_stored":39078605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":121916,"kb_used_data":82408,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94249924,"statfs":{"total":96636764160,"available":96511922176,"internally_reserved":0,"allocated":84385792,"data_stored":98473140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":185812,"kb_used_data":146300,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94186028,"statfs":{"total":96636764160,"available":96446492672,"internally_reserved":0,"allocated":149811200,"data_stored":184002346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":142796,"kb_used_data":95032,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94229044,"statfs":{"total":96636764160,"available":96490541056,"internally_reserved":0,"allocated":97312768,"data_stored":112411864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49448,"kb_used_data":14080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322392,"statfs":{"total":96636764160,"available":96586129408,"internally_reserved":0,"allocated":14417920,"data_stored":13853517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77076,"kb_used_data":37548,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94294764,"statfs":{"total":96636764160,"available":96557838336,"internally_reserved":0,"allocated":38449152,"data_stored":39411226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113180,"kb_used_data":86132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94258660,"statfs":{"total":96636764160,"available":96520867840,"internally_reserved":0,"allocated":88199168,"data_stored":104742598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86728704,"data_stored":103755652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37568512,"data_stored":39014629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13541376,"data_stored":13456939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":96436224,"data_stored":112015286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":148922368,"data_stored":183599947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":83505152,"data_stored":98076497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32071680,"data_stored":38719948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":99844096,"data_stored":116081071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:20:42.292 INFO:tasks.thrashosds.thrasher:Setting 3.a to [4, 1] 2024-09-12T07:20:42.292 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.a', '4', '1'] 2024-09-12T07:20:42.292 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.a 4 1 2024-09-12T07:20:42.365 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:42.367+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:42.375 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:42.375 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:42.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:42.450 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone32 from 0x1f12dfb (overlap 0x1f12dfb) to 0x15c0696 2024-09-12T07:20:42.466 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:42.468+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:42.506 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:42.507 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:42.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:42.587 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-12T07:20:42.587 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:42.589+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:42.598 INFO:teuthology.orchestra.run.smithi046.stdout:4244 clone 34 order 18 su 65536 sc 4 2024-09-12T07:20:42.669 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:42.670+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:42.669 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:42.669 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:42.678 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:42.769 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:42.771+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:42.826 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:42.826 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:42.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:42.878 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:42.880+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:42.942 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:42.943 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:42.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:42.972 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:42.973+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:43.097 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:43.074+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:43.099 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:43.099 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:43.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:43.192 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:43.194+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:43.255 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:43.255 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:43.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:43.304 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:43.306+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:set 3.a pg_upmap mapping to [4,1] 2024-09-12T07:20:43.375 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:43.377+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:43.386 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:43.386 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:43.395 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:43.397 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T07:20:43.476 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:43.478+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:43.544 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:43.544 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:43.553 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:43.596 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:43.598+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:43.660 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:43.660 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:43.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:43.711 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:43.712+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:43.778 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:43.780+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:43.817 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:43.817 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:43.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:43.879 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:43.880+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:43.933 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:43.933 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:43.941 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:43.982 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:43.984+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:44.081 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:44.082+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:44.088 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:44.089 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:44.097 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:44.181 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:44.183+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:44.248 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:44.248 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:44.256 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:44.274 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:44.319 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:44.320+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:44.356 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is set 2024-09-12T07:20:44.366 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:44.366 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:44.375 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:44.415 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:44.418 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:44.420+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:44.433 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:44.449 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:44.465 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:44.466 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:44.481 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:44.481 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:44.482 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:44.482 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:44.484 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:44.486+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:44.491 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:44.498 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:44.499 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:44.515 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:44.516 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:44.532 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:44.532 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:44.585 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:44.587+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:44.637 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:44.638 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:44.638 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:44.638 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:44.646 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:44.686 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:44.688+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:44.753 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:44.753 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:44.753 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:44.753 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:44.753 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:44.762 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:44.806 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:44.808+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:44.869 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321501232999999, 2024-09-12T07:20:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:20:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.870 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:20:44.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321480696000002, 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:20:44.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321460692999999, 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.879 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:44.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.882 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.883 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321440744, 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:20:44.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:20:44.887 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321419876, 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.888 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:20:44.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:20:44.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321398174999999, 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:20:44.892 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.896 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321377734999999, 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:20:44.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:20:44.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321350013999997, 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.901 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:44.905 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321327079, 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.909 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321306778999997, 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:20:44.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:20:44.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321286503000003, 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.914 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.915 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.917 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.918 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.919 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:44.919 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321266344000001, 2024-09-12T07:20:44.919 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:20:44.919 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.919 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.919 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.919 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.919 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.919 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.919 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:20:44.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321129145999997, 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:20:44.923 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.924 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:20:44.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.321106497999999, 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:20:44.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.929 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68088 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 393216~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.314663738999997, 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10346986399999999, 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.932 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68088 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555923+0000", 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.891e-06 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555929+0000", 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0549999999999996e-06 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555934+0000", 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6419999999999998e-06 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659060+0000", 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103125862 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659139+0000", 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9450000000000007e-05 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659391+0000", 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025158400000000001 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.314644512000001, 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.937 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:20:44.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:20:44.940 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.314414874999997, 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.941 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:20:44.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.945 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.307491184, 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:44.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:20:44.949 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.307131601000002, 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.950 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.951 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:20:44.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:44.954 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.307051614999999, 2024-09-12T07:20:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:20:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:20:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.955 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:20:44.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:20:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:20:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:44.959 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:44.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:44.961 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:44.962+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:45.061 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:45.063+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:45.077 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:45.077 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:45.077 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:45.078 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:45.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:45.171 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:45.173+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:45.235 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:45.235 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:45.244 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:45.285 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:45.286+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:45.363 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:45.365+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:45.394 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:45.394 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82749 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 429056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.75076567, 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055052166, 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.395 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82749 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:45.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0800000000000001e-07 2024-09-12T07:20:45.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644459+0000", 2024-09-12T07:20:45.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1769999999999999e-06 2024-09-12T07:20:45.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644462+0000", 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8779999999999998e-06 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644465+0000", 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6509999999999999e-06 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699279+0000", 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054813832999999999 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699359+0000", 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0150999999999999e-05 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699510+0000", 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015026800000000001 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.399 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82750 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 432128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.750752786, 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055234722, 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82750 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644471+0000", 2024-09-12T07:20:45.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.17e-07 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644472+0000", 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.19e-06 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644482+0000", 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0332e-05 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644489+0000", 2024-09-12T07:20:45.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7039999999999997e-06 2024-09-12T07:20:45.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699549+0000", 2024-09-12T07:20:45.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055060588000000001 2024-09-12T07:20:45.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699592+0000", 2024-09-12T07:20:45.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2707000000000003e-05 2024-09-12T07:20:45.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699705+0000", 2024-09-12T07:20:45.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112984 2024-09-12T07:20:45.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.404 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.405 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82751 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 436224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.405 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:45.405 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.750725432000003, 2024-09-12T07:20:45.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055415691000000003, 2024-09-12T07:20:45.405 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82751 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644846+0000", 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034807600000000001 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644850+0000", 2024-09-12T07:20:45.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1649999999999999e-06 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644860+0000", 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6949999999999995e-06 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644866+0000", 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3640000000000004e-06 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699735+0000", 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054868888999999997 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699776+0000", 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0722000000000001e-05 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699913+0000", 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013778 2024-09-12T07:20:45.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82752 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 439296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.750347689000002, 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226917, 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82752 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:45.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644878+0000", 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7850000000000001e-06 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644882+0000", 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9990000000000002e-06 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644885+0000", 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.965e-06 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699952+0000", 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067243000000002 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699989+0000", 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7450999999999998e-05 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700102+0000", 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113124 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82753 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 443392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.750335032999999, 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055422102000000001, 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82753 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645016+0000", 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000127847 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645024+0000", 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7419999999999997e-06 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645030+0000", 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7229999999999998e-06 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645034+0000", 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9380000000000002e-06 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700132+0000", 2024-09-12T07:20:45.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055097699 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700171+0000", 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9093e-05 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700310+0000", 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013905999999999999 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82754 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 446464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.750179125999999, 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055424703999999998, 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82754 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.419 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645318+0000", 2024-09-12T07:20:45.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027397499999999998 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645322+0000", 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.766e-06 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645331+0000", 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2140000000000002e-06 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645338+0000", 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0929999999999997e-06 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700330+0000", 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054991384999999997 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700369+0000", 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:20:45.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700469+0000", 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9913000000000004e-05 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82755 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 449536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.749874136000003, 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055388918000000002, 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.423 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82755 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645500+0000", 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015050999999999999 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645502+0000", 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-12T07:20:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645509+0000", 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3969999999999999e-06 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645513+0000", 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9230000000000004e-06 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700507+0000", 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054994692999999997 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700583+0000", 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5656999999999997e-05 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700738+0000", 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154869 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.427 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82756 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 453632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.749703248000003, 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055451746000000003, 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82756 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645763+0000", 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000243105 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645766+0000", 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0750000000000002e-06 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645774+0000", 2024-09-12T07:20:45.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9370000000000004e-06 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645783+0000", 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5159999999999994e-06 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700780+0000", 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054997329999999997 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700835+0000", 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4789e-05 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700972+0000", 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013699400000000001 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82757 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.432 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.749428879999996, 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055382590000000002, 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82757 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645926+0000", 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131979 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645929+0000", 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0199999999999999e-06 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645935+0000", 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645939+0000", 2024-09-12T07:20:45.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.924e-06 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701007+0000", 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067374000000002 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701043+0000", 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6325999999999997e-05 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701177+0000", 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013387300000000001 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82758 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 460800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.749271487000001, 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055417890999999997, 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.437 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82758 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:45.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646164+0000", 2024-09-12T07:20:45.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000212189 2024-09-12T07:20:45.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646167+0000", 2024-09-12T07:20:45.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-12T07:20:45.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646173+0000", 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4139999999999999e-06 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646177+0000", 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701208+0000", 2024-09-12T07:20:45.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055031322000000001 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701254+0000", 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5701999999999997e-05 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701370+0000", 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115456 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82759 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 463872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.749038925999997, 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055376838999999997, 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82759 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.442 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646368+0000", 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183399 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646372+0000", 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9190000000000001e-06 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646380+0000", 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7819999999999999e-06 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646384+0000", 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701407+0000", 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055022763000000002 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701455+0000", 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7660000000000001e-05 2024-09-12T07:20:45.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701561+0000", 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010642899999999999 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82760 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.748831377000002, 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055355572999999998, 2024-09-12T07:20:45.446 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82760 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646631+0000", 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000238798 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646634+0000", 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.016e-06 2024-09-12T07:20:45.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646640+0000", 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2750000000000001e-06 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646644+0000", 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7859999999999998e-06 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701599+0000", 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054955693999999999 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701638+0000", 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8319999999999999e-05 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701747+0000", 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109684 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.450 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82761 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.748572748000001, 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055296990999999997, 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82761 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646756+0000", 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105899 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646759+0000", 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8380000000000002e-06 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646765+0000", 2024-09-12T07:20:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646769+0000", 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7900000000000001e-06 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701785+0000", 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055015888999999998 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701832+0000", 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6514000000000002e-05 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701947+0000", 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115833 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82762 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 474112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.748447181000003, 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055346863000000003, 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82762 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647063+0000", 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000287115 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647066+0000", 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0189999999999998e-06 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647075+0000", 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7099999999999996e-06 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647081+0000", 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6790000000000002e-06 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701977+0000", 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054896665999999997 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702015+0000", 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.79e-05 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702123+0000", 2024-09-12T07:20:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107774 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82763 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.748132263999999, 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226509999999999, 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82763 2024-09-12T07:20:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647236+0000", 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000144597 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647239+0000", 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1329999999999999e-06 2024-09-12T07:20:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647245+0000", 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647249+0000", 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2270000000000004e-06 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702160+0000", 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054910549000000003 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702205+0000", 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5575000000000003e-05 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702317+0000", 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011194899999999999 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.464 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82764 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 481280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.747966513000001, 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055229250000000001, 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.465 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82764 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647471+0000", 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000214625 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647474+0000", 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8509999999999999e-06 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647480+0000", 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.28e-06 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647484+0000", 2024-09-12T07:20:45.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.771e-06 2024-09-12T07:20:45.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702346+0000", 2024-09-12T07:20:45.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054861551000000001 2024-09-12T07:20:45.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702382+0000", 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.65e-05 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702486+0000", 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010367200000000001 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82765 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 485376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.747730965999999, 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055184729000000002, 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82765 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647681+0000", 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188965 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647693+0000", 2024-09-12T07:20:45.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.225e-05 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647700+0000", 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9929999999999998e-06 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647704+0000", 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9469999999999996e-06 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702522+0000", 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054817346000000003 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702566+0000", 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4672999999999997e-05 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702677+0000", 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110555 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.473 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82766 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 488448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.747511727999999, 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055132717999999997, 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82766 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647959+0000", 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024739599999999999 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647964+0000", 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9409999999999998e-06 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647975+0000", 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1404e-05 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647983+0000", 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2969999999999998e-06 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702706+0000", 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054723690999999998 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702743+0000", 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6464999999999997e-05 2024-09-12T07:20:45.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702844+0000", 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101524 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82767 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 492544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.747227686999999, 2024-09-12T07:20:45.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055054448999999998, 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82767 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648182+0000", 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018652299999999999 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648185+0000", 2024-09-12T07:20:45.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3289999999999998e-06 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648192+0000", 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7159999999999997e-06 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648196+0000", 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1439999999999996e-06 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702873+0000", 2024-09-12T07:20:45.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054676356000000002 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702930+0000", 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7643999999999999e-05 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703050+0000", 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119737 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:148945 3.10 3:0cac0a45:::rbd_data.1160161ce0c8.000000000000009d:head [delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9614217259999993, 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89878390500000005, 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 148945 2024-09-12T07:20:45.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433803+0000", 2024-09-12T07:20:45.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.164e-06 2024-09-12T07:20:45.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433807+0000", 2024-09-12T07:20:45.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5000000000000001e-06 2024-09-12T07:20:45.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433816+0000", 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1859999999999998e-06 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433822+0000", 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5300000000000004e-06 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433886+0000", 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4510000000000004e-05 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433896+0000", 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0012e-05 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329359+0000", 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89546257399999996 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329585+0000", 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022641800000000001 2024-09-12T07:20:45.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:20:45.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329712+0000", 2024-09-12T07:20:45.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126339 2024-09-12T07:20:45.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:45.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.330704+0000", 2024-09-12T07:20:45.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099219900000000003 2024-09-12T07:20:45.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332513+0000", 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001808689 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332544+0000", 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1186999999999998e-05 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332585+0000", 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1597000000000002e-05 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.490 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.490 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:45.491 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:45.492 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:45.493+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:45.592 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:45.593+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:45.640 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:45.640 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:45.641 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:45.641 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:45.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:45.692 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:45.694+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:45.793 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:45.794+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:45.796 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:45.797 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:45.797 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:45.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:45.894 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:45.895+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:45.955 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:45.955 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:45.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:45.955 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:45.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.955 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:45.955 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:45.955 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.572205621999998, 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:20:45.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.509127358000001, 2024-09-12T07:20:45.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:20:45.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.505422273000001, 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:20:45.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:20:45.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:20:45.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:20:45.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:20:45.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:20:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.507075993999997, 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:20:45.973 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:20:45.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:20:45.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:20:45.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116552 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5267456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.487746642000001, 2024-09-12T07:20:45.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034412801999999999, 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116552 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0400000000000002e-07 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467850+0000", 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6529999999999998e-06 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467856+0000", 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467861+0000", 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502080+0000", 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034218947999999999 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502159+0000", 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9121000000000004e-05 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502260+0000", 2024-09-12T07:20:45.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100846 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116553 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5472256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.487725943000001, 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034609677999999998, 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.983 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.984 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.984 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116553 2024-09-12T07:20:45.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.984 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:20:45.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467868+0000", 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6800000000000001e-07 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467870+0000", 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.773e-06 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467876+0000", 2024-09-12T07:20:45.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0079999999999999e-06 2024-09-12T07:20:45.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467880+0000", 2024-09-12T07:20:45.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-12T07:20:45.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502304+0000", 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423924000000002 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502379+0000", 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5010000000000002e-05 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502477+0000", 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7942999999999994e-05 2024-09-12T07:20:45.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116554 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5308416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.487706403000001, 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034779583000000003, 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116554 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467889+0000", 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7430000000000001e-06 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467895+0000", 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8159999999999999e-06 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467900+0000", 2024-09-12T07:20:45.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-12T07:20:45.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502507+0000", 2024-09-12T07:20:45.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034607252999999998 2024-09-12T07:20:45.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502580+0000", 2024-09-12T07:20:45.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3337999999999995e-05 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502667+0000", 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6468e-05 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116555 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5349376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.487687429000001, 2024-09-12T07:20:45.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034965729000000001, 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116555 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1899999999999998e-07 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467908+0000", 2024-09-12T07:20:45.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5740000000000001e-06 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467914+0000", 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9939999999999997e-06 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467918+0000", 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.622e-06 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502696+0000", 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034777798999999998 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502780+0000", 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3551e-05 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502872+0000", 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.187e-05 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:45.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116557 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3506176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.486593015, 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034335510999999999, 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116557 2024-09-12T07:20:45.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.000 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:20:46.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:20:46.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469001+0000", 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469003+0000", 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469012+0000", 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8030000000000003e-06 2024-09-12T07:20:46.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469019+0000", 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6959999999999999e-06 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503153+0000", 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034134096000000003 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503236+0000", 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2886000000000006e-05 2024-09-12T07:20:46.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503336+0000", 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9919000000000003e-05 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116558 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3469312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.486562734, 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034519743999999998, 2024-09-12T07:20:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116558 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469034+0000", 2024-09-12T07:20:46.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469042+0000", 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469047+0000", 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503370+0000", 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034323201999999997 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503454+0000", 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4109000000000004e-05 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503550+0000", 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6009999999999997e-05 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116559 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3551232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.486535902, 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034689659999999997, 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116559 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.008 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469058+0000", 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469061+0000", 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.497e-06 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469069+0000", 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2579999999999995e-06 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469074+0000", 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1279999999999999e-06 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503582+0000", 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034508351 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503656+0000", 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3234999999999999e-05 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503747+0000", 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1638999999999995e-05 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116560 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.486509173999998, 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034902912000000001, 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.012 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116560 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469085+0000", 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4700000000000001e-07 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469087+0000", 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4760000000000001e-06 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469095+0000", 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2339999999999994e-06 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469100+0000", 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503796+0000", 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034695411000000002 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503868+0000", 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2032000000000004e-05 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503987+0000", 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119369 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.016 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116561 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3592192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.486485774999998, 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035102170000000002, 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116561 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469110+0000", 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1629999999999999e-06 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469116+0000", 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.251e-06 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469121+0000", 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8210000000000001e-06 2024-09-12T07:20:46.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504017+0000", 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034895772999999998 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504108+0000", 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1027999999999997e-05 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504210+0000", 2024-09-12T07:20:46.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101729 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116562 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.486464931, 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035285642999999998, 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.021 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116562 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469129+0000", 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469131+0000", 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1550000000000001e-06 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469138+0000", 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5520000000000001e-06 2024-09-12T07:20:46.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469142+0000", 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9099999999999996e-06 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504239+0000", 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.03509669 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504311+0000", 2024-09-12T07:20:46.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1650999999999996e-05 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504414+0000", 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103286 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116563 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.486444297999999, 2024-09-12T07:20:46.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035471691, 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116563 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469153+0000", 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.05e-06 2024-09-12T07:20:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469158+0000", 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.874e-06 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469163+0000", 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8069999999999999e-06 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504444+0000", 2024-09-12T07:20:46.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035280793999999997 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504523+0000", 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9036999999999996e-05 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504621+0000", 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7743000000000003e-05 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116564 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5435392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.486423187, 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035734657000000003, 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116564 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.030 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469171+0000", 2024-09-12T07:20:46.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469172+0000", 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.832e-06 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469179+0000", 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1449999999999996e-06 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469184+0000", 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504721+0000", 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035536890000000002 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504809+0000", 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7229000000000001e-05 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504905+0000", 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6337999999999998e-05 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116565 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5476352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.486402520999999, 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035933158, 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116565 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469193+0000", 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.973e-06 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469201+0000", 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3730000000000004e-06 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469206+0000", 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1660000000000002e-06 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504937+0000", 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035731205000000002 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505035+0000", 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7759999999999999e-05 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505124+0000", 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9258000000000005e-05 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116566 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5271552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:46.038 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.486379934999999, 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036116848999999999, 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116566 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:20:46.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469214+0000", 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469216+0000", 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8759999999999999e-06 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469222+0000", 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.173e-06 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469227+0000", 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3070000000000002e-06 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505154+0000", 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035926362000000003 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505239+0000", 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5261999999999998e-05 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505330+0000", 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1436999999999999e-05 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.042 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116567 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5312512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.486359405999998, 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036305875000000001, 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.043 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116567 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.58e-07 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469242+0000", 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469249+0000", 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5549999999999999e-06 2024-09-12T07:20:46.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469254+0000", 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9899999999999997e-06 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505361+0000", 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036106923999999999 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505449+0000", 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8531999999999999e-05 2024-09-12T07:20:46.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505540+0000", 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0805999999999998e-05 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116568 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5353472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.486332733000001, 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036561383000000003, 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116568 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469263+0000", 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7549999999999999e-06 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469269+0000", 2024-09-12T07:20:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7219999999999996e-06 2024-09-12T07:20:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469273+0000", 2024-09-12T07:20:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:20:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505631+0000", 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036357997000000003 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505711+0000", 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9331000000000003e-05 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505822+0000", 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000111208 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.051 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.052 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.052 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:46.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:46.054 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:46.055+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:46.111 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone33 from 0x1f12dfb (overlap 0x1f12dfb) to 0x1bf762e 2024-09-12T07:20:46.153 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:46.155+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:46.180 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:46.180 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:46.180 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:46.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:46.255 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:46.256+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:46.279 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-12T07:20:46.307 INFO:teuthology.orchestra.run.smithi046.stdout:4245 read 0x15d5172 thru 0x15d52ee (0x17d bytes) 2024-09-12T07:20:46.314 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:46.314 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:46.315 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:46.315 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:46.315 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:46.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:46.330 INFO:teuthology.orchestra.run.smithi046.stdout:4246 write 0xccd661c thru 0xccdab51 (0x4536 bytes) 2024-09-12T07:20:46.333 INFO:teuthology.orchestra.run.smithi046.stdout:4247 punch from 0x9d5ba15 to 0x9d5fe02, (0x43ed bytes) 2024-09-12T07:20:46.333 INFO:teuthology.orchestra.run.smithi046.stdout:4248 punch from 0x672d200 to 0x67399d2, (0xc7d2 bytes) 2024-09-12T07:20:46.333 INFO:teuthology.orchestra.run.smithi046.stdout:4250 writesame 0x249bb83 thru 0x249ee45 data_size 0x17(0x32c3 bytes) 2024-09-12T07:20:46.336 INFO:teuthology.orchestra.run.smithi046.stdout:4251 writesame 0x7e8aed7 thru 0x7e91ead data_size 0x1(0x6fd7 bytes) 2024-09-12T07:20:46.349 INFO:teuthology.orchestra.run.smithi046.stdout:4252 punch from 0x43841da to 0x4386e5d, (0x2c83 bytes) 2024-09-12T07:20:46.349 INFO:teuthology.orchestra.run.smithi046.stdout:4253 write 0x45b31d4 thru 0x45b34d6 (0x303 bytes) 2024-09-12T07:20:46.353 INFO:teuthology.orchestra.run.smithi046.stdout:4254 read 0xbe172bf thru 0xbe1f0b5 (0x7df7 bytes) 2024-09-12T07:20:46.353 INFO:teuthology.orchestra.run.smithi046.stdout:4256 read 0x1892ce4 thru 0x1895f3b (0x3258 bytes) 2024-09-12T07:20:46.356 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:46.358+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:46.357 INFO:teuthology.orchestra.run.smithi046.stdout:4258 trunc from 0xccdab52 to 0x9cdc160 2024-09-12T07:20:46.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:20:46.373 INFO:teuthology.orchestra.run.smithi046.stdout:4259 writesame 0x71aec3a thru 0x71bbfc7 data_size 0xd(0xd38e bytes) 2024-09-12T07:20:46.443 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:46.443 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:46.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:46.443 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:46.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.443 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77440 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13156352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.443 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:46.443 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.198334383999999, 2024-09-12T07:20:46.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051399724000000001, 2024-09-12T07:20:46.444 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77440 2024-09-12T07:20:46.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.444 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246267+0000", 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082256199999999999 2024-09-12T07:20:46.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246271+0000", 2024-09-12T07:20:46.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6700000000000002e-06 2024-09-12T07:20:46.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246283+0000", 2024-09-12T07:20:46.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.207e-05 2024-09-12T07:20:46.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246291+0000", 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296645+0000", 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050353872000000001 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296729+0000", 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4215000000000006e-05 2024-09-12T07:20:46.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296844+0000", 2024-09-12T07:20:46.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114626 2024-09-12T07:20:46.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.448 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.448 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77446 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.448 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.197335869, 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051543643, 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77446 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246579+0000", 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013651399999999999 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246583+0000", 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1509999999999997e-06 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246593+0000", 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5670000000000001e-06 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246599+0000", 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5189999999999998e-06 2024-09-12T07:20:46.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297826+0000", 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051226381000000001 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297903+0000", 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6710000000000002e-05 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297986+0000", 2024-09-12T07:20:46.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3801000000000006e-05 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77447 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 5783552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.197168255000001, 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051512637, 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77447 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246812+0000", 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000201506 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246817+0000", 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246828+0000", 2024-09-12T07:20:46.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1693e-05 2024-09-12T07:20:46.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246836+0000", 2024-09-12T07:20:46.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4429999999999997e-06 2024-09-12T07:20:46.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298021+0000", 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051184940999999998 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298067+0000", 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6292000000000003e-05 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298123+0000", 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5828999999999997e-05 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.457 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77448 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.196928517000003, 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051506024999999997, 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77448 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246892+0000", 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1854999999999998e-05 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246895+0000", 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3129999999999999e-06 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246904+0000", 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6179999999999995e-06 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246911+0000", 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7939999999999997e-06 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298164+0000", 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051253559999999997 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298249+0000", 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4549000000000007e-05 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298356+0000", 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107336 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77449 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.196857321000003, 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051582830000000003, 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77449 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247137+0000", 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021599499999999999 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247142+0000", 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247154+0000", 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1878e-05 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247168+0000", 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4114000000000001e-05 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298391+0000", 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051223249999999998 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298446+0000", 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4366000000000001e-05 2024-09-12T07:20:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298504+0000", 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8273000000000003e-05 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77450 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12681216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.196597750999999, 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051562375000000001, 2024-09-12T07:20:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77450 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2799999999999996e-07 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247184+0000", 2024-09-12T07:20:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247192+0000", 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5920000000000003e-06 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247197+0000", 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6720000000000001e-06 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298550+0000", 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051352082 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298641+0000", 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1236000000000005e-05 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298743+0000", 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102428 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77451 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7061504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.196573637, 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051813773, 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77451 2024-09-12T07:20:46.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.473 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:46.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:46.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247335+0000", 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013012599999999999 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247340+0000", 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.544e-06 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247351+0000", 2024-09-12T07:20:46.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.111e-05 2024-09-12T07:20:46.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247358+0000", 2024-09-12T07:20:46.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9859999999999997e-06 2024-09-12T07:20:46.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298909+0000", 2024-09-12T07:20:46.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051551781999999997 2024-09-12T07:20:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298960+0000", 2024-09-12T07:20:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0594999999999999e-05 2024-09-12T07:20:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299019+0000", 2024-09-12T07:20:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8629999999999999e-05 2024-09-12T07:20:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77452 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.196408857000002, 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051892134999999999, 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.477 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77452 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247535+0000", 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165508 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247552+0000", 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6514000000000001e-05 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247565+0000", 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3372e-05 2024-09-12T07:20:46.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247577+0000", 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299064+0000", 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051486697999999997 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299150+0000", 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6549000000000001e-05 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299262+0000", 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011135 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77453 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.196188136000004, 2024-09-12T07:20:46.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051901623000000001, 2024-09-12T07:20:46.482 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77453 2024-09-12T07:20:46.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.482 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:46.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:46.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247710+0000", 2024-09-12T07:20:46.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011951 2024-09-12T07:20:46.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247713+0000", 2024-09-12T07:20:46.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4539999999999999e-06 2024-09-12T07:20:46.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247720+0000", 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0339999999999999e-06 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247730+0000", 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0030000000000001e-05 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299297+0000", 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051566446000000002 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299382+0000", 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4963000000000006e-05 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299492+0000", 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011018600000000001 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77454 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7618560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.486 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:46.487 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.196039589000002, 2024-09-12T07:20:46.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051921851999999998, 2024-09-12T07:20:46.487 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.487 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77454 2024-09-12T07:20:46.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.487 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:46.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:46.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247900+0000", 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016088699999999999 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247904+0000", 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6950000000000002e-06 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247912+0000", 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7129999999999994e-06 2024-09-12T07:20:46.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247921+0000", 2024-09-12T07:20:46.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0299999999999999e-06 2024-09-12T07:20:46.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299526+0000", 2024-09-12T07:20:46.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051604744000000001 2024-09-12T07:20:46.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299601+0000", 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5129000000000005e-05 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299661+0000", 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9654000000000002e-05 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.491 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77455 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12746752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.195845720999998, 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051971254000000001, 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77455 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:46.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248181+0000", 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024790000000000001 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248185+0000", 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5079999999999999e-06 2024-09-12T07:20:46.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248197+0000", 2024-09-12T07:20:46.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.132e-05 2024-09-12T07:20:46.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248203+0000", 2024-09-12T07:20:46.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6320000000000002e-06 2024-09-12T07:20:46.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299705+0000", 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051501342999999998 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299799+0000", 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4340999999999997e-05 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299904+0000", 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010521 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.496 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77456 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7979008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.195563495999998, 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051860577999999997, 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77456 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248545+0000", 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033009700000000001 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248550+0000", 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248560+0000", 2024-09-12T07:20:46.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248568+0000", 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4939999999999997e-06 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299973+0000", 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051405470000000002 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300019+0000", 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5448999999999998e-05 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300076+0000", 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7073000000000001e-05 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77457 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13172736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.501 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.195197168, 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051737914000000003, 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77457 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:46.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248582+0000", 2024-09-12T07:20:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:20:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248584+0000", 2024-09-12T07:20:46.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:20:46.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248593+0000", 2024-09-12T07:20:46.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-12T07:20:46.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248609+0000", 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5664000000000001e-05 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300117+0000", 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051507605999999997 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300212+0000", 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4919000000000004e-05 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300319+0000", 2024-09-12T07:20:46.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010776399999999999 2024-09-12T07:20:46.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.506 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.506 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77458 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.507 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:46.507 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.19515475, 2024-09-12T07:20:46.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051863578, 2024-09-12T07:20:46.507 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.507 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77458 2024-09-12T07:20:46.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.507 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248627+0000", 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1389999999999999e-06 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248635+0000", 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4019999999999996e-06 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248641+0000", 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1689999999999997e-06 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300359+0000", 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051717889000000003 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300418+0000", 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8689e-05 2024-09-12T07:20:46.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300487+0000", 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9756000000000001e-05 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77459 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13238272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.195129057000003, 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052076504000000003, 2024-09-12T07:20:46.511 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77459 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250611+0000", 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019617879999999999 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250618+0000", 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1829999999999997e-06 2024-09-12T07:20:46.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250629+0000", 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0438000000000001e-05 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250648+0000", 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9355e-05 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300528+0000", 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049879252999999998 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300619+0000", 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1910000000000006e-05 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300726+0000", 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010657700000000001 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.515 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77467 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.192923237999999, 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05160754, 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77467 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250856+0000", 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250858+0000", 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9180000000000001e-06 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250864+0000", 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.066e-06 2024-09-12T07:20:46.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250869+0000", 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302283+0000", 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051414591000000003 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302378+0000", 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4283000000000005e-05 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302463+0000", 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5249999999999999e-05 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77468 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14909440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:46.520 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.192903665000003, 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051742162000000001, 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77468 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250877+0000", 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.815e-06 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250883+0000", 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250888+0000", 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302496+0000", 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051607394000000001 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302553+0000", 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6971000000000001e-05 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302617+0000", 2024-09-12T07:20:46.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4418999999999994e-05 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77469 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14974976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.192883485999999, 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965694, 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77469 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:46.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:46.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:20:46.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250897+0000", 2024-09-12T07:20:46.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9130000000000001e-06 2024-09-12T07:20:46.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250903+0000", 2024-09-12T07:20:46.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0129999999999999e-06 2024-09-12T07:20:46.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250908+0000", 2024-09-12T07:20:46.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9679999999999997e-06 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302661+0000", 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051753066 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302755+0000", 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3376999999999997e-05 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302861+0000", 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105924 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77470 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12828672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.192863809000002, 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052098589000000001, 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77470 2024-09-12T07:20:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250917+0000", 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8190000000000001e-06 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250923+0000", 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250928+0000", 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9420000000000003e-06 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302894+0000", 2024-09-12T07:20:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965863000000001 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302955+0000", 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1235999999999994e-05 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303013+0000", 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8167000000000001e-05 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77471 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12894208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.192844262999998, 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052326263999999997, 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:46.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77471 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256961+0000", 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060271209999999999 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256972+0000", 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0755e-05 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256986+0000", 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845e-05 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256996+0000", 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303060+0000", 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.046063661999999998 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303149+0000", 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8949999999999994e-05 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:46.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:46.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:46.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303261+0000", 2024-09-12T07:20:46.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011195 2024-09-12T07:20:46.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.539 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:46.539 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:46.539 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:46.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:46.541 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:46.542+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:46.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:20:46.641 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:46.643+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:46.692 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:46.693 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:46.693 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:46.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:46.707 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:46.708 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:46.727 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:46.727 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:46.742 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:46.744+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:46.743 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:46.761 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:46.778 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:46.796 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:46.812 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:46.813 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:46.813 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:46.813 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:46.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:46.829 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:46.830 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:46.881 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:46.883+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:46.936 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:46.936 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:46.943 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:46.944+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:46.945 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:47.044 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:47.046+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:47.058 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:47.059 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:47.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:47.151 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:47.153+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:47.216 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:47.217 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:47.224 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:47.266 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:47.267+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:47.347 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:47.349+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:47.369 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:20:47.372 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:47.372 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:47.382 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:47.384 INFO:teuthology.orchestra.run.smithi046.stdout:4260 trunc from 0x9cdc160 to 0xb4807d6 2024-09-12T07:20:47.391 INFO:teuthology.orchestra.run.smithi046.stdout:4261 punch from 0x506fe6f to 0x507ab07, (0xac98 bytes) 2024-09-12T07:20:47.391 INFO:teuthology.orchestra.run.smithi046.stdout:4262 read 0xb383654 thru 0xb38dbe9 (0xa596 bytes) 2024-09-12T07:20:47.391 INFO:teuthology.orchestra.run.smithi046.stdout:4263 read 0x7ed16c5 thru 0x7ee01e1 (0xeb1d bytes) 2024-09-12T07:20:47.391 INFO:teuthology.orchestra.run.smithi046.stdout:4264 clone 35 order 25 su 4194304 sc 14 2024-09-12T07:20:47.448 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:47.449+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:47.489 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:47.489 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:47.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:47.581 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:47.583+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:47.645 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:47.646 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:47.649 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:47.650+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:47.653 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:47.750 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:47.751+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:47.800 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:47.800 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:47.809 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:47.850 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:47.852+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:47.951 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:47.953+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:47.956 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:47.956 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:47.965 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:48.053 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:48.055+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:48.113 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:48.113 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:48.121 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:48.166 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:48.168+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:48.229 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:48.229 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:48.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:48.255 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:48.256+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:48.355 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:48.357+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:48.364 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:20:48.364 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-12T07:20:48.389 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:48.389 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:48.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:48.456 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:48.457+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:48.489 INFO:teuthology.orchestra.run.smithi046.stdout:flattening image image_client.0-clone34 2024-09-12T07:20:48.519 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:48.519 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:48.529 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:48.556 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:48.558+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:48.636 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:48.636 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:48.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:48.689 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:48.690+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:48.696 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-12T07:20:48.734 INFO:teuthology.orchestra.run.smithi046.stdout:4265 read 0x86d85a6 thru 0x86e2581 (0x9fdc bytes) 2024-09-12T07:20:48.739 INFO:teuthology.orchestra.run.smithi046.stdout:4266 trunc from 0xb4807d6 to 0x662a4db 2024-09-12T07:20:48.758 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:48.760+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:48.769 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:48.769 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:48.780 INFO:teuthology.orchestra.run.smithi046.stdout:4268 writesame 0xe574216 thru 0xe5826ac data_size 0x1(0xe497 bytes) 2024-09-12T07:20:48.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:48.782 INFO:teuthology.orchestra.run.smithi046.stdout:4269 read 0x6275b05 thru 0x627af13 (0x540f bytes) 2024-09-12T07:20:48.782 INFO:teuthology.orchestra.run.smithi046.stdout:4272 read 0x39762d3 thru 0x397c5f3 (0x6321 bytes) 2024-09-12T07:20:48.783 INFO:teuthology.orchestra.run.smithi046.stdout:4274 write 0x83543cf thru 0x835fe1a (0xba4c bytes) 2024-09-12T07:20:48.784 INFO:teuthology.orchestra.run.smithi046.stdout:4275 clone 36 order 20 su 16384 sc 2 2024-09-12T07:20:48.841 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:48.859 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:48.860+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:48.889 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:48.890 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:48.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:48.960 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:48.961+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:48.968 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:48.969 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:48.985 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:48.986 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:49.002 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:49.012 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:49.012 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:49.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:49.012 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:49.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.012 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82749 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 429056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.367776380000002, 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055052166, 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82749 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0800000000000001e-07 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644459+0000", 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1769999999999999e-06 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644462+0000", 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8779999999999998e-06 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644465+0000", 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6509999999999999e-06 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699279+0000", 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054813832999999999 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699359+0000", 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0150999999999999e-05 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699510+0000", 2024-09-12T07:20:49.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015026800000000001 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82750 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 432128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.367763496000002, 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055234722, 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82750 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644471+0000", 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.17e-07 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644472+0000", 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.19e-06 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644482+0000", 2024-09-12T07:20:49.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0332e-05 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644489+0000", 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7039999999999997e-06 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699549+0000", 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055060588000000001 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699592+0000", 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2707000000000003e-05 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699705+0000", 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112984 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82751 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 436224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:49.021 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.367736141999998, 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055415691000000003, 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82751 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:49.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644846+0000", 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034807600000000001 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644850+0000", 2024-09-12T07:20:49.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1649999999999999e-06 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644860+0000", 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6949999999999995e-06 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644866+0000", 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3640000000000004e-06 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699735+0000", 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054868888999999997 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699776+0000", 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0722000000000001e-05 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699913+0000", 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013778 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82752 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 439296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.367358398999997, 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226917, 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82752 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644878+0000", 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7850000000000001e-06 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644882+0000", 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9990000000000002e-06 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644885+0000", 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.965e-06 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699952+0000", 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067243000000002 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699989+0000", 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7450999999999998e-05 2024-09-12T07:20:49.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700102+0000", 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113124 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82753 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 443392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.367345743000001, 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055422102000000001, 2024-09-12T07:20:49.030 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82753 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645016+0000", 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000127847 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645024+0000", 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7419999999999997e-06 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645030+0000", 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7229999999999998e-06 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645034+0000", 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9380000000000002e-06 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700132+0000", 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055097699 2024-09-12T07:20:49.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700171+0000", 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9093e-05 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700310+0000", 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013905999999999999 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82754 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 446464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.367189836000001, 2024-09-12T07:20:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055424703999999998, 2024-09-12T07:20:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82754 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645318+0000", 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027397499999999998 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645322+0000", 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.766e-06 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645331+0000", 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2140000000000002e-06 2024-09-12T07:20:49.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645338+0000", 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0929999999999997e-06 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700330+0000", 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054991384999999997 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700369+0000", 2024-09-12T07:20:49.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700469+0000", 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9913000000000004e-05 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82755 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 449536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.366884845999998, 2024-09-12T07:20:49.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055388918000000002, 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82755 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645500+0000", 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015050999999999999 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645502+0000", 2024-09-12T07:20:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645509+0000", 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3969999999999999e-06 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645513+0000", 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9230000000000004e-06 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700507+0000", 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054994692999999997 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700583+0000", 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5656999999999997e-05 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700738+0000", 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154869 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82756 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 453632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.366713957999998, 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055451746000000003, 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82756 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645763+0000", 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000243105 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645766+0000", 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0750000000000002e-06 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645774+0000", 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9370000000000004e-06 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645783+0000", 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5159999999999994e-06 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700780+0000", 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054997329999999997 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700835+0000", 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4789e-05 2024-09-12T07:20:49.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700972+0000", 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013699400000000001 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82757 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.366439589999999, 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055382590000000002, 2024-09-12T07:20:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82757 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645926+0000", 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131979 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645929+0000", 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0199999999999999e-06 2024-09-12T07:20:49.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645935+0000", 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645939+0000", 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.924e-06 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701007+0000", 2024-09-12T07:20:49.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067374000000002 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701043+0000", 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6325999999999997e-05 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701177+0000", 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013387300000000001 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.053 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82758 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 460800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.053 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:49.053 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.366282196999997, 2024-09-12T07:20:49.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055417890999999997, 2024-09-12T07:20:49.056 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82758 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646164+0000", 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000212189 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646167+0000", 2024-09-12T07:20:49.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646173+0000", 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4139999999999999e-06 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646177+0000", 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701208+0000", 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055031322000000001 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701254+0000", 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5701999999999997e-05 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701370+0000", 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115456 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82759 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 463872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.366049636, 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055376838999999997, 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82759 2024-09-12T07:20:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646368+0000", 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183399 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646372+0000", 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9190000000000001e-06 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646380+0000", 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7819999999999999e-06 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646384+0000", 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701407+0000", 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055022763000000002 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701455+0000", 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7660000000000001e-05 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701561+0000", 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010642899999999999 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82760 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.365842086999997, 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055355572999999998, 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82760 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646631+0000", 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000238798 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646634+0000", 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.016e-06 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646640+0000", 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2750000000000001e-06 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646644+0000", 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7859999999999998e-06 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701599+0000", 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054955693999999999 2024-09-12T07:20:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701638+0000", 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8319999999999999e-05 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701747+0000", 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109684 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82761 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.365583458000003, 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055296990999999997, 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82761 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646756+0000", 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105899 2024-09-12T07:20:49.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646759+0000", 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8380000000000002e-06 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646765+0000", 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646769+0000", 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7900000000000001e-06 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701785+0000", 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055015888999999998 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701832+0000", 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6514000000000002e-05 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701947+0000", 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115833 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82762 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 474112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.365457890999998, 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055346863000000003, 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82762 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:20:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647063+0000", 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000287115 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647066+0000", 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0189999999999998e-06 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647075+0000", 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7099999999999996e-06 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647081+0000", 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6790000000000002e-06 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701977+0000", 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054896665999999997 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702015+0000", 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.79e-05 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702123+0000", 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107774 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82763 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.365142974000001, 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226509999999999, 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82763 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647236+0000", 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000144597 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647239+0000", 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1329999999999999e-06 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647245+0000", 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647249+0000", 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2270000000000004e-06 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702160+0000", 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054910549000000003 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702205+0000", 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5575000000000003e-05 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702317+0000", 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011194899999999999 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82764 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 481280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.364977222999997, 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055229250000000001, 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82764 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647471+0000", 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000214625 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647474+0000", 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8509999999999999e-06 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647480+0000", 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.28e-06 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647484+0000", 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.771e-06 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702346+0000", 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054861551000000001 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702382+0000", 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.65e-05 2024-09-12T07:20:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702486+0000", 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010367200000000001 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82765 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 485376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.364741676000001, 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055184729000000002, 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82765 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647681+0000", 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188965 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647693+0000", 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.225e-05 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647700+0000", 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9929999999999998e-06 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647704+0000", 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9469999999999996e-06 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702522+0000", 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054817346000000003 2024-09-12T07:20:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702566+0000", 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4672999999999997e-05 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702677+0000", 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110555 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82766 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 488448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.364522438000002, 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055132717999999997, 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82766 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647959+0000", 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024739599999999999 2024-09-12T07:20:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647964+0000", 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9409999999999998e-06 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647975+0000", 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1404e-05 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647983+0000", 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2969999999999998e-06 2024-09-12T07:20:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702706+0000", 2024-09-12T07:20:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054723690999999998 2024-09-12T07:20:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702743+0000", 2024-09-12T07:20:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6464999999999997e-05 2024-09-12T07:20:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702844+0000", 2024-09-12T07:20:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101524 2024-09-12T07:20:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82767 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 492544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:20:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.364238397000001, 2024-09-12T07:20:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055054448999999998, 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82767 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648182+0000", 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018652299999999999 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648185+0000", 2024-09-12T07:20:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3289999999999998e-06 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648192+0000", 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7159999999999997e-06 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648196+0000", 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1439999999999996e-06 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702873+0000", 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054676356000000002 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702930+0000", 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7643999999999999e-05 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703050+0000", 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119737 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:148945 3.10 3:0cac0a45:::rbd_data.1160161ce0c8.000000000000009d:head [delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.578432436, 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89878390500000005, 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 148945 2024-09-12T07:20:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433803+0000", 2024-09-12T07:20:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.164e-06 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433807+0000", 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5000000000000001e-06 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433816+0000", 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1859999999999998e-06 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433822+0000", 2024-09-12T07:20:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5300000000000004e-06 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433886+0000", 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4510000000000004e-05 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433896+0000", 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0012e-05 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329359+0000", 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89546257399999996 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329585+0000", 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022641800000000001 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329712+0000", 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126339 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.330704+0000", 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099219900000000003 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332513+0000", 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001808689 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332544+0000", 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1186999999999998e-05 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332585+0000", 2024-09-12T07:20:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1597000000000002e-05 2024-09-12T07:20:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.107 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:49.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:49.109 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:49.110 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:49.110 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:49.111 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:49.111 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:49.111 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:49.112 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:49.112 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:49.110+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:49.209 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:49.211+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:49.256 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:49.256 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:49.256 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:49.256 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:49.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:49.310 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:49.311+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:49.372 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:49.372 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:49.372 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:49.372 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:49.372 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:49.375 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.0 to 1 (10000) 2024-09-12T07:20:49.384 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:49.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:20:49.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-12T07:20:49.435 INFO:teuthology.orchestra.run.smithi046.stdout:leaving image image_client.0-clone35 intact 2024-09-12T07:20:49.435 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:49.435+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:49.511 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:49.513+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:49.549 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:49.549 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.166098458999997, 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:20:49.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:20:49.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.103020194999999, 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:20:49.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:20:49.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:20:49.561 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.099315109999999, 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.562 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:20:49.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.567 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.100968831000003, 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:20:49.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116552 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5267456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.081639479, 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034412801999999999, 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116552 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0400000000000002e-07 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467850+0000", 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6529999999999998e-06 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467856+0000", 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467861+0000", 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502080+0000", 2024-09-12T07:20:49.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034218947999999999 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502159+0000", 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9121000000000004e-05 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502260+0000", 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100846 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116553 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5472256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.081618779999999, 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034609677999999998, 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116553 2024-09-12T07:20:49.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467868+0000", 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6800000000000001e-07 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467870+0000", 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.773e-06 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467876+0000", 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0079999999999999e-06 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467880+0000", 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502304+0000", 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423924000000002 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502379+0000", 2024-09-12T07:20:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5010000000000002e-05 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502477+0000", 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7942999999999994e-05 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116554 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5308416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.081599239999999, 2024-09-12T07:20:49.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034779583000000003, 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116554 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467889+0000", 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7430000000000001e-06 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467895+0000", 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8159999999999999e-06 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467900+0000", 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502507+0000", 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034607252999999998 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502580+0000", 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3337999999999995e-05 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502667+0000", 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6468e-05 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.586 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116555 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5349376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.081580266, 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034965729000000001, 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116555 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1899999999999998e-07 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467908+0000", 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5740000000000001e-06 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467914+0000", 2024-09-12T07:20:49.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9939999999999997e-06 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467918+0000", 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.622e-06 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502696+0000", 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034777798999999998 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502780+0000", 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3551e-05 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502872+0000", 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.187e-05 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116557 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3506176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:20:49.591 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.080485851999999, 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034335510999999999, 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116557 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469001+0000", 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469003+0000", 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469012+0000", 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8030000000000003e-06 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469019+0000", 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6959999999999999e-06 2024-09-12T07:20:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503153+0000", 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034134096000000003 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503236+0000", 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2886000000000006e-05 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503336+0000", 2024-09-12T07:20:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9919000000000003e-05 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116558 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3469312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.080455571000002, 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034519743999999998, 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.596 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116558 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469034+0000", 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469042+0000", 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469047+0000", 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503370+0000", 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034323201999999997 2024-09-12T07:20:49.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503454+0000", 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4109000000000004e-05 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503550+0000", 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6009999999999997e-05 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116559 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3551232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:20:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.080428738999998, 2024-09-12T07:20:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034689659999999997, 2024-09-12T07:20:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116559 2024-09-12T07:20:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:20:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469058+0000", 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469061+0000", 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.497e-06 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469069+0000", 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2579999999999995e-06 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469074+0000", 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1279999999999999e-06 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503582+0000", 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034508351 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503656+0000", 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3234999999999999e-05 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503747+0000", 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1638999999999995e-05 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.605 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116560 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.080402011, 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034902912000000001, 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116560 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469085+0000", 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4700000000000001e-07 2024-09-12T07:20:49.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469087+0000", 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4760000000000001e-06 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469095+0000", 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2339999999999994e-06 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469100+0000", 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503796+0000", 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034695411000000002 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503868+0000", 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2032000000000004e-05 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503987+0000", 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119369 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116561 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3592192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.080378612000001, 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035102170000000002, 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116561 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469110+0000", 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1629999999999999e-06 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469116+0000", 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.251e-06 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469121+0000", 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8210000000000001e-06 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504017+0000", 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034895772999999998 2024-09-12T07:20:49.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504108+0000", 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1027999999999997e-05 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504210+0000", 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101729 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116562 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.080357767999999, 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035285642999999998, 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116562 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469129+0000", 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:20:49.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469131+0000", 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1550000000000001e-06 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469138+0000", 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5520000000000001e-06 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469142+0000", 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9099999999999996e-06 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504239+0000", 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.03509669 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504311+0000", 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1650999999999996e-05 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504414+0000", 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103286 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116563 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.080337135000001, 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035471691, 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116563 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469153+0000", 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.05e-06 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469158+0000", 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.874e-06 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469163+0000", 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8069999999999999e-06 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504444+0000", 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035280793999999997 2024-09-12T07:20:49.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504523+0000", 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9036999999999996e-05 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504621+0000", 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7743000000000003e-05 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116564 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5435392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.080316023999998, 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035734657000000003, 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116564 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.624 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469171+0000", 2024-09-12T07:20:49.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469172+0000", 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.832e-06 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469179+0000", 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1449999999999996e-06 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469184+0000", 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504721+0000", 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035536890000000002 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504809+0000", 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7229000000000001e-05 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504905+0000", 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6337999999999998e-05 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116565 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5476352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.080295358000001, 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035933158, 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.628 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116565 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469193+0000", 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.973e-06 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469201+0000", 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3730000000000004e-06 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469206+0000", 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1660000000000002e-06 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504937+0000", 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035731205000000002 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505035+0000", 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7759999999999999e-05 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505124+0000", 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9258000000000005e-05 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116566 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5271552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.080272772000001, 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036116848999999999, 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116566 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469214+0000", 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:20:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469216+0000", 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8759999999999999e-06 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469222+0000", 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.173e-06 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469227+0000", 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3070000000000002e-06 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505154+0000", 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035926362000000003 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505239+0000", 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5261999999999998e-05 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505330+0000", 2024-09-12T07:20:49.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1436999999999999e-05 2024-09-12T07:20:49.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.637 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.637 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116567 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5312512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.637 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.080252243, 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036305875000000001, 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116567 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.58e-07 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469242+0000", 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469249+0000", 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5549999999999999e-06 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469254+0000", 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9899999999999997e-06 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505361+0000", 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036106923999999999 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505449+0000", 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8531999999999999e-05 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505540+0000", 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0805999999999998e-05 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.641 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116568 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5353472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.08022557, 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036561383000000003, 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:49.642 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116568 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469263+0000", 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7549999999999999e-06 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469269+0000", 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7219999999999996e-06 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469273+0000", 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505631+0000", 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036357997000000003 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505711+0000", 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9331000000000003e-05 2024-09-12T07:20:49.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:49.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:49.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:49.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505822+0000", 2024-09-12T07:20:49.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000111208 2024-09-12T07:20:49.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.646 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:49.646 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:49.646 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:49.648 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:49.648 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-12T07:20:49.648 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:49.649+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:49.749 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:49.750+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:49.801 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:49.801 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:49.810 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:49.813 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:49.816 INFO:teuthology.orchestra.run.smithi046.stdout:4277 trunc from 0xe5826ad to 0xb44df93 2024-09-12T07:20:49.850 INFO:teuthology.orchestra.run.smithi046.stdout:4279 punch from 0xadda71e to 0xaddcdfc, (0x26de bytes) 2024-09-12T07:20:49.850 INFO:teuthology.orchestra.run.smithi046.stdout:4280 punch from 0x66cab92 to 0x66d7e96, (0xd304 bytes) 2024-09-12T07:20:49.850 INFO:teuthology.orchestra.run.smithi046.stdout:4281 read 0x39606bc thru 0x3970541 (0xfe86 bytes) 2024-09-12T07:20:49.851 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:49.851+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:49.851 INFO:teuthology.orchestra.run.smithi046.stdout:4282 write 0x66a11bc thru 0x66a7dd4 (0x6c19 bytes) 2024-09-12T07:20:49.883 INFO:teuthology.orchestra.run.smithi046.stdout:4283 writesame 0xd31deea thru 0xd32299b data_size 0x3(0x4ab2 bytes) 2024-09-12T07:20:49.883 INFO:teuthology.orchestra.run.smithi046.stdout:4284 write 0xa79598b thru 0xa796a88 (0x10fe bytes) 2024-09-12T07:20:49.910 INFO:teuthology.orchestra.run.smithi046.stdout:4285 trunc from 0xd32299c to 0x166ab2f 2024-09-12T07:20:49.943 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:49.943 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:49.944 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:49.944 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:49.944 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:49.951 INFO:teuthology.orchestra.run.smithi046.stdout:4286 trunc from 0x166ab2f to 0x25e57a 2024-09-12T07:20:49.951 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:49.953+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:49.953 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:49.979 INFO:teuthology.orchestra.run.smithi046.stdout:4287 punch from 0x118a53 to 0x126cca, (0xe277 bytes) 2024-09-12T07:20:49.994 INFO:teuthology.orchestra.run.smithi046.stdout:4288 writesame 0x90087ee thru 0x9017e31 data_size 0x2(0xf644 bytes) 2024-09-12T07:20:49.994 INFO:teuthology.orchestra.run.smithi046.stdout:4289 read 0x59f7e7e thru 0x5a04cbe (0xce41 bytes) 2024-09-12T07:20:49.995 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:49.995 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:50.000 INFO:teuthology.orchestra.run.smithi046.stdout:4294 write 0x913c950 thru 0x9145132 (0x87e3 bytes) 2024-09-12T07:20:50.004 INFO:teuthology.orchestra.run.smithi046.stdout:4295 writesame 0x73f36b4 thru 0x73f4cf8 data_size 0x1(0x1645 bytes) 2024-09-12T07:20:50.011 INFO:teuthology.orchestra.run.smithi046.stdout:4296 read 0x4093b20 thru 0x409f8e1 (0xbdc2 bytes) 2024-09-12T07:20:50.011 INFO:teuthology.orchestra.run.smithi046.stdout:4297 read 0x661a7f1 thru 0x66204ba (0x5cca bytes) 2024-09-12T07:20:50.021 INFO:teuthology.orchestra.run.smithi046.stdout:4298 writesame 0xcc78d1b thru 0xcc7f2e8 data_size 0x53(0x65ce bytes) 2024-09-12T07:20:50.022 INFO:teuthology.orchestra.run.smithi046.stdout:4299 read 0xc225d61 thru 0xc22f684 (0x9924 bytes) 2024-09-12T07:20:50.022 INFO:teuthology.orchestra.run.smithi046.stdout:4301 write 0x10e9f80 thru 0x10efc8d (0x5d0e bytes) 2024-09-12T07:20:50.034 INFO:teuthology.orchestra.run.smithi046.stdout:4302 trunc from 0xcc7f2e9 to 0x6d3813e 2024-09-12T07:20:50.045 INFO:teuthology.orchestra.run.smithi046.stdout:4303 read 0xe2119 thru 0xe6127 (0x400f bytes) 2024-09-12T07:20:50.045 INFO:teuthology.orchestra.run.smithi046.stdout:4304 read 0x595b671 thru 0x596054c (0x4edc bytes) 2024-09-12T07:20:50.045 INFO:teuthology.orchestra.run.smithi046.stdout:4305 read 0x463e377 thru 0x46434e3 (0x516d bytes) 2024-09-12T07:20:50.046 INFO:teuthology.orchestra.run.smithi046.stdout:4306 read 0xca7236 thru 0xca9927 (0x26f2 bytes) 2024-09-12T07:20:50.048 INFO:teuthology.orchestra.run.smithi046.stdout:4308 trunc from 0x6d3813e to 0x47f2a26 2024-09-12T07:20:50.052 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:50.054+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:50.054 INFO:teuthology.orchestra.run.smithi046.stdout:4310 write 0x197e96c thru 0x198e965 (0xfffa bytes) 2024-09-12T07:20:50.058 INFO:teuthology.orchestra.run.smithi046.stdout:4311 writesame 0x2d9bcf2 thru 0x2da2970 data_size 0x37(0x6c7f bytes) 2024-09-12T07:20:50.062 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:50.078 INFO:teuthology.orchestra.run.smithi046.stdout:4313 write 0xdab0dac thru 0xdabf780 (0xe9d5 bytes) 2024-09-12T07:20:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:4315 punch from 0x5486632 to 0x5492939, (0xc307 bytes) 2024-09-12T07:20:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:4316 read 0xc9bdc09 thru 0xc9c74f6 (0x98ee bytes) 2024-09-12T07:20:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:4317 punch from 0x85c2376 to 0x85ccd44, (0xa9ce bytes) 2024-09-12T07:20:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:4320 read 0x865f54d thru 0x86685f3 (0x90a7 bytes) 2024-09-12T07:20:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:4321 write 0x92f9da9 thru 0x930776e (0xd9c6 bytes) 2024-09-12T07:20:50.086 INFO:teuthology.orchestra.run.smithi046.stdout:4322 writesame 0x5386b3f thru 0x5392880 data_size 0x6(0xbd42 bytes) 2024-09-12T07:20:50.092 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:50.092 INFO:teuthology.orchestra.run.smithi046.stdout:4326 writesame 0x5b97888 thru 0x5b9cc92 data_size 0xd(0x540b bytes) 2024-09-12T07:20:50.097 INFO:teuthology.orchestra.run.smithi046.stdout:4328 write 0xa17b98c thru 0xa18b669 (0xfcde bytes) 2024-09-12T07:20:50.101 INFO:teuthology.orchestra.run.smithi046.stdout:4329 writesame 0xbbfea9a thru 0xbc0801b data_size 0x3(0x9582 bytes) 2024-09-12T07:20:50.112 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:50.112 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:50.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:50.112 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:50.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.112 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77440 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13156352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.112 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:50.112 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.867127328999999, 2024-09-12T07:20:50.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051399724000000001, 2024-09-12T07:20:50.112 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77440 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246267+0000", 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082256199999999999 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246271+0000", 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6700000000000002e-06 2024-09-12T07:20:50.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246283+0000", 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.207e-05 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246291+0000", 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296645+0000", 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050353872000000001 2024-09-12T07:20:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296729+0000", 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4215000000000006e-05 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296844+0000", 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114626 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77446 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.866128814, 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051543643, 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77446 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.117 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246579+0000", 2024-09-12T07:20:50.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013651399999999999 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246583+0000", 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1509999999999997e-06 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246593+0000", 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5670000000000001e-06 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246599+0000", 2024-09-12T07:20:50.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5189999999999998e-06 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297826+0000", 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051226381000000001 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297903+0000", 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6710000000000002e-05 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297986+0000", 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3801000000000006e-05 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77447 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 5783552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.865961200000001, 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051512637, 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77447 2024-09-12T07:20:50.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.122 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:50.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246812+0000", 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000201506 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246817+0000", 2024-09-12T07:20:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:50.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246828+0000", 2024-09-12T07:20:50.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1693e-05 2024-09-12T07:20:50.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246836+0000", 2024-09-12T07:20:50.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4429999999999997e-06 2024-09-12T07:20:50.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298021+0000", 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051184940999999998 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298067+0000", 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6292000000000003e-05 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298123+0000", 2024-09-12T07:20:50.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5828999999999997e-05 2024-09-12T07:20:50.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.126 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.126 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77448 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.126 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:50.126 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.865721462000003, 2024-09-12T07:20:50.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051506024999999997, 2024-09-12T07:20:50.126 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77448 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246892+0000", 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1854999999999998e-05 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246895+0000", 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3129999999999999e-06 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246904+0000", 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6179999999999995e-06 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246911+0000", 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7939999999999997e-06 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298164+0000", 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051253559999999997 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298249+0000", 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4549000000000007e-05 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298356+0000", 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107336 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77449 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.130 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.865650266000003, 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051582830000000003, 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77449 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247137+0000", 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021599499999999999 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247142+0000", 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247154+0000", 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1878e-05 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247168+0000", 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4114000000000001e-05 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298391+0000", 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051223249999999998 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298446+0000", 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4366000000000001e-05 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298504+0000", 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8273000000000003e-05 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.134 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77450 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12681216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.865390695999999, 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051562375000000001, 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77450 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2799999999999996e-07 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247184+0000", 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247192+0000", 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5920000000000003e-06 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247197+0000", 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6720000000000001e-06 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298550+0000", 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051352082 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298641+0000", 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1236000000000005e-05 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298743+0000", 2024-09-12T07:20:50.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102428 2024-09-12T07:20:50.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.139 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77451 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7061504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.865366582, 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051813773, 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77451 2024-09-12T07:20:50.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247335+0000", 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013012599999999999 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247340+0000", 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.544e-06 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247351+0000", 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.111e-05 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247358+0000", 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9859999999999997e-06 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298909+0000", 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051551781999999997 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298960+0000", 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0594999999999999e-05 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299019+0000", 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8629999999999999e-05 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77452 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.865201802000001, 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051892134999999999, 2024-09-12T07:20:50.144 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77452 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247535+0000", 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165508 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247552+0000", 2024-09-12T07:20:50.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6514000000000001e-05 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247565+0000", 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3372e-05 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247577+0000", 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299064+0000", 2024-09-12T07:20:50.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051486697999999997 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299150+0000", 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6549000000000001e-05 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299262+0000", 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011135 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77453 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.864981081000003, 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051901623000000001, 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77453 2024-09-12T07:20:50.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.150 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:50.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:50.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247710+0000", 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011951 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247713+0000", 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4539999999999999e-06 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247720+0000", 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0339999999999999e-06 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247730+0000", 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0030000000000001e-05 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299297+0000", 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051566446000000002 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299382+0000", 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4963000000000006e-05 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299492+0000", 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011018600000000001 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.153 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77454 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7618560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.864832534000001, 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051921851999999998, 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77454 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247900+0000", 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016088699999999999 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247904+0000", 2024-09-12T07:20:50.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6950000000000002e-06 2024-09-12T07:20:50.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247912+0000", 2024-09-12T07:20:50.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7129999999999994e-06 2024-09-12T07:20:50.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247921+0000", 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0299999999999999e-06 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299526+0000", 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051604744000000001 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299601+0000", 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5129000000000005e-05 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299661+0000", 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9654000000000002e-05 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77455 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12746752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.864638665999998, 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051971254000000001, 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77455 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248181+0000", 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024790000000000001 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248185+0000", 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5079999999999999e-06 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248197+0000", 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.132e-05 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248203+0000", 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6320000000000002e-06 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299705+0000", 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051501342999999998 2024-09-12T07:20:50.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299799+0000", 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4340999999999997e-05 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299904+0000", 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010521 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77456 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7979008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.864356440999998, 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051860577999999997, 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77456 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248545+0000", 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033009700000000001 2024-09-12T07:20:50.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248550+0000", 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248560+0000", 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248568+0000", 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4939999999999997e-06 2024-09-12T07:20:50.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299973+0000", 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051405470000000002 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300019+0000", 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5448999999999998e-05 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300076+0000", 2024-09-12T07:20:50.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7073000000000001e-05 2024-09-12T07:20:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77457 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13172736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.863990113, 2024-09-12T07:20:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051737914000000003, 2024-09-12T07:20:50.167 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77457 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248582+0000", 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248584+0000", 2024-09-12T07:20:50.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248593+0000", 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248609+0000", 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5664000000000001e-05 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300117+0000", 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051507605999999997 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300212+0000", 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4919000000000004e-05 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300319+0000", 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010776399999999999 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.171 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77458 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.863947695, 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051863578, 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77458 2024-09-12T07:20:50.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248627+0000", 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1389999999999999e-06 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248635+0000", 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4019999999999996e-06 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248641+0000", 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1689999999999997e-06 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300359+0000", 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051717889000000003 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300418+0000", 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8689e-05 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300487+0000", 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9756000000000001e-05 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77459 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13238272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.863922002000002, 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052076504000000003, 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.176 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77459 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250611+0000", 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019617879999999999 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250618+0000", 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1829999999999997e-06 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250629+0000", 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0438000000000001e-05 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250648+0000", 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9355e-05 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300528+0000", 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049879252999999998 2024-09-12T07:20:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300619+0000", 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1910000000000006e-05 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300726+0000", 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010657700000000001 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77467 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.861716182999999, 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05160754, 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77467 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250856+0000", 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:20:50.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250858+0000", 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9180000000000001e-06 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250864+0000", 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.066e-06 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250869+0000", 2024-09-12T07:20:50.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302283+0000", 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051414591000000003 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302378+0000", 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4283000000000005e-05 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302463+0000", 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5249999999999999e-05 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77468 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14909440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.861696610000003, 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051742162000000001, 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.185 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77468 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:50.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250877+0000", 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.815e-06 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250883+0000", 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250888+0000", 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302496+0000", 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051607394000000001 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302553+0000", 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6971000000000001e-05 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302617+0000", 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4418999999999994e-05 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.189 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77469 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14974976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.861676430999999, 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965694, 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77469 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250897+0000", 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9130000000000001e-06 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250903+0000", 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0129999999999999e-06 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250908+0000", 2024-09-12T07:20:50.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9679999999999997e-06 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302661+0000", 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051753066 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302755+0000", 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3376999999999997e-05 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302861+0000", 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105924 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77470 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12828672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.861656754000002, 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052098589000000001, 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77470 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250917+0000", 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8190000000000001e-06 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250923+0000", 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250928+0000", 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9420000000000003e-06 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302894+0000", 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965863000000001 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302955+0000", 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1235999999999994e-05 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303013+0000", 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8167000000000001e-05 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.198 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77471 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12894208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.861637207999998, 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052326263999999997, 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77471 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256961+0000", 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060271209999999999 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256972+0000", 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0755e-05 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256986+0000", 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845e-05 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256996+0000", 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:50.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303060+0000", 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.046063661999999998 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303149+0000", 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8949999999999994e-05 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303261+0000", 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011195 2024-09-12T07:20:50.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.203 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.203 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:50.204 INFO:teuthology.orchestra.run.smithi046.stdout:4330 trunc from 0xdabf781 to 0x35478ee 2024-09-12T07:20:50.205 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:4332 writesame 0xb6cfcb thru 0xb7c271 data_size 0x1(0xf2a7 bytes) 2024-09-12T07:20:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:4334 write 0xea87988 thru 0xea967da (0xee53 bytes) 2024-09-12T07:20:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:4335 writesame 0xd544bea thru 0xd54ae50 data_size 0x1b(0x6267 bytes) 2024-09-12T07:20:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:4336 read 0x72360fb thru 0x723ea5b (0x8961 bytes) 2024-09-12T07:20:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:4337 write 0x469eafc thru 0x46a1b9a (0x309f bytes) 2024-09-12T07:20:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:4339 write 0xe7973f thru 0xe81898 (0x815a bytes) 2024-09-12T07:20:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:4340 read 0x84b093e thru 0x84ba108 (0x97cb bytes) 2024-09-12T07:20:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:4341 read 0x5c57c95 thru 0x5c58914 (0xc80 bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4342 read 0x1b79724 thru 0x1b83337 (0x9c14 bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4343 read 0x8af500 thru 0x8b8241 (0x8d42 bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4344 punch from 0x5c81511 to 0x5c895a8, (0x8097 bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4345 read 0x4d29771 thru 0x4d2adea (0x167a bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4347 read 0xb83ff30 thru 0xb84710a (0x71db bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4348 write 0x96f1daf thru 0x96f8893 (0x6ae5 bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4350 write 0x8c4b767 thru 0x8c5061d (0x4eb7 bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4351 read 0x85b21ab thru 0x85b4f15 (0x2d6b bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4353 write 0x7243f60 thru 0x724c387 (0x8428 bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4354 read 0xdcc3c9c thru 0xdccb309 (0x766e bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4356 read 0xda4d7af thru 0xda57b84 (0xa3d6 bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4358 read 0xd74c2e2 thru 0xd755066 (0x8d85 bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4361 write 0x6411c38 thru 0x641de27 (0xc1f0 bytes) 2024-09-12T07:20:50.208 INFO:teuthology.orchestra.run.smithi046.stdout:4362 writesame 0x5d79178 thru 0x5d7f94a data_size 0x1(0x67d3 bytes) 2024-09-12T07:20:50.209 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:50.205+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:50.213 INFO:teuthology.orchestra.run.smithi046.stdout:4363 writesame 0xe543b15 thru 0xe54a219 data_size 0x3b(0x6705 bytes) 2024-09-12T07:20:50.217 INFO:teuthology.orchestra.run.smithi046.stdout:4364 punch from 0x45adf10 to 0x45b5bd3, (0x7cc3 bytes) 2024-09-12T07:20:50.217 INFO:teuthology.orchestra.run.smithi046.stdout:4367 read 0x984324d thru 0x984b3c6 (0x817a bytes) 2024-09-12T07:20:50.218 INFO:teuthology.orchestra.run.smithi046.stdout:4368 writesame 0x8ab0635 thru 0x8ab56f8 data_size 0x6(0x50c4 bytes) 2024-09-12T07:20:50.222 INFO:teuthology.orchestra.run.smithi046.stdout:4370 write 0x266153c thru 0x26686f7 (0x71bc bytes) 2024-09-12T07:20:50.226 INFO:teuthology.orchestra.run.smithi046.stdout:4371 read 0x71fbf59 thru 0x7208329 (0xc3d1 bytes) 2024-09-12T07:20:50.227 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:50.227 INFO:teuthology.orchestra.run.smithi046.stdout:4374 write 0x1ab3907 thru 0x1ab4b34 (0x122e bytes) 2024-09-12T07:20:50.231 INFO:teuthology.orchestra.run.smithi046.stdout:4375 punch from 0x77a2938 to 0x77aa142, (0x780a bytes) 2024-09-12T07:20:50.231 INFO:teuthology.orchestra.run.smithi046.stdout:4376 punch from 0x7c99577 to 0x7c9d84b, (0x42d4 bytes) 2024-09-12T07:20:50.246 INFO:teuthology.orchestra.run.smithi046.stdout:4377 trunc from 0xea967db to 0x130b3a5 2024-09-12T07:20:50.273 INFO:teuthology.orchestra.run.smithi046.stdout:4378 trunc from 0x130b3a5 to 0xa951f59 2024-09-12T07:20:50.279 INFO:teuthology.orchestra.run.smithi046.stdout:4379 flatten 2024-09-12T07:20:50.305 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:50.306+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:50.316 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:50.316 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:50.317 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:50.324 INFO:teuthology.orchestra.run.smithi046.stdout:4383 writesame 0xc7a8f31 thru 0xc7b53ec data_size 0x12(0xc4bc bytes) 2024-09-12T07:20:50.324 INFO:teuthology.orchestra.run.smithi046.stdout:4384 read 0x2a1ad5 thru 0x2b17b7 (0xfce3 bytes) 2024-09-12T07:20:50.336 INFO:teuthology.orchestra.run.smithi046.stdout:4386 trunc from 0xc7b53ed to 0x2921064 2024-09-12T07:20:50.339 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:156751 3.2 3:46ad04d0:::rbd_object_map.1160f231e22:head [call lock.assert_locked in=44b,call rbd.object_map_resize in=29b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T07:20:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:50.340537+0000", 2024-09-12T07:20:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0013051460000000001, 2024-09-12T07:20:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001322503, 2024-09-12T07:20:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "waiting for sub ops", 2024-09-12T07:20:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 156751 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.340537+0000", 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.340537+0000", 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.340537+0000", 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1800000000000005e-07 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.340542+0000", 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2030000000000002e-06 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.340551+0000", 2024-09-12T07:20:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7950000000000002e-06 2024-09-12T07:20:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.340557+0000", 2024-09-12T07:20:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.344e-06 2024-09-12T07:20:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.340600+0000", 2024-09-12T07:20:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3619999999999999e-05 2024-09-12T07:20:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.340660+0000", 2024-09-12T07:20:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9809000000000003e-05 2024-09-12T07:20:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-12T07:20:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.340819+0000", 2024-09-12T07:20:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015908200000000001 2024-09-12T07:20:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:20:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.341476+0000", 2024-09-12T07:20:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00065682399999999997 2024-09-12T07:20:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4448.0:156751 3.2 e88/46)", 2024-09-12T07:20:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:50.341830+0000", 2024-09-12T07:20:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.2061e-05, 2024-09-12T07:20:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9231000000000002e-05, 2024-09-12T07:20:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.341830+0000", 2024-09-12T07:20:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.341830+0000", 2024-09-12T07:20:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.341830+0000", 2024-09-12T07:20:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4099999999999998e-07 2024-09-12T07:20:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.341833+0000", 2024-09-12T07:20:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.23e-06 2024-09-12T07:20:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.341842+0000", 2024-09-12T07:20:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1720000000000007e-06 2024-09-12T07:20:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.341848+0000", 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.489e-06 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.341893+0000", 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5058000000000001e-05 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:50.341899+0000", 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2149999999999998e-06 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:20:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 2 2024-09-12T07:20:50.350 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:50.361 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:50.363 INFO:teuthology.orchestra.run.smithi046.stdout:4388 write 0xeba0638 thru 0xeba1d7f (0x1748 bytes) 2024-09-12T07:20:50.366 INFO:teuthology.orchestra.run.smithi046.stdout:4390 punch from 0x427069e to 0x42737d2, (0x3134 bytes) 2024-09-12T07:20:50.367 INFO:teuthology.orchestra.run.smithi046.stdout:4392 write 0x673b4bd thru 0x6741091 (0x5bd5 bytes) 2024-09-12T07:20:50.372 INFO:teuthology.orchestra.run.smithi046.stdout:4393 write 0xc2080be thru 0xc215a9f (0xd9e2 bytes) 2024-09-12T07:20:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.1 to 1 (10000) 2024-09-12T07:20:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:20:50.397 INFO:teuthology.orchestra.run.smithi046.stdout:4394 trunc from 0xeba1d80 to 0xc16af4 2024-09-12T07:20:50.406 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:50.407+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:50.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-12T07:20:50.413 INFO:teuthology.orchestra.run.smithi046.stdout:4395 read 0x71a598 thru 0x724817 (0xa280 bytes) 2024-09-12T07:20:50.415 INFO:teuthology.orchestra.run.smithi046.stdout:4396 trunc from 0xc16af4 to 0x163a39f 2024-09-12T07:20:50.436 INFO:teuthology.orchestra.run.smithi046.stdout:4397 writesame 0xa74e654 thru 0xa74fec6 data_size 0x1(0x1873 bytes) 2024-09-12T07:20:50.441 INFO:teuthology.orchestra.run.smithi046.stdout:4398 trunc from 0xa74fec7 to 0xec7b3ac 2024-09-12T07:20:50.446 INFO:teuthology.orchestra.run.smithi046.stdout:4399 read 0x4da0f96 thru 0x4db0761 (0xf7cc bytes) 2024-09-12T07:20:50.447 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:50.447 INFO:teuthology.orchestra.run.smithi046.stdout:4401 punch from 0x16c4fed to 0x16ce855, (0x9868 bytes) 2024-09-12T07:20:50.447 INFO:teuthology.orchestra.run.smithi046.stdout:4402 read 0x16f1f79 thru 0x16f5055 (0x30dd bytes) 2024-09-12T07:20:50.447 INFO:teuthology.orchestra.run.smithi046.stdout:4403 write 0xaa2ab73 thru 0xaa3916e (0xe5fc bytes) 2024-09-12T07:20:50.462 INFO:teuthology.orchestra.run.smithi046.stdout:4405 trunc from 0xec7b3ac to 0x523c442 2024-09-12T07:20:50.473 INFO:teuthology.orchestra.run.smithi046.stdout:4407 punch from 0x127551e to 0x12801e9, (0xaccb bytes) 2024-09-12T07:20:50.473 INFO:teuthology.orchestra.run.smithi046.stdout:4409 read 0x25c4731 thru 0x25c6ed7 (0x27a7 bytes) 2024-09-12T07:20:50.484 INFO:teuthology.orchestra.run.smithi046.stdout:4410 writesame 0xa0e32a5 thru 0xa0f1608 data_size 0x84(0xe364 bytes) 2024-09-12T07:20:50.491 INFO:teuthology.orchestra.run.smithi046.stdout:4412 write 0xb5a9a91 thru 0xb5ad4e4 (0x3a54 bytes) 2024-09-12T07:20:50.497 INFO:teuthology.orchestra.run.smithi046.stdout:4413 read 0x8e089ef thru 0x8e08ad9 (0xeb bytes) 2024-09-12T07:20:50.497 INFO:teuthology.orchestra.run.smithi046.stdout:4415 read 0x48f82a7 thru 0x48fa26f (0x1fc9 bytes) 2024-09-12T07:20:50.497 INFO:teuthology.orchestra.run.smithi046.stdout:4416 write 0x3110221 thru 0x31143fa (0x41da bytes) 2024-09-12T07:20:50.499 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:50.499 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:50.499 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:50.499 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:50.499 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:50.500 INFO:teuthology.orchestra.run.smithi046.stdout:4418 punch from 0x7db6f03 to 0x7dc0c6a, (0x9d67 bytes) 2024-09-12T07:20:50.500 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:50.500 INFO:teuthology.orchestra.run.smithi046.stdout:4420 write 0x8b1eca3 thru 0x8b277bf (0x8b1d bytes) 2024-09-12T07:20:50.503 INFO:teuthology.orchestra.run.smithi046.stdout:4421 writesame 0x4404d40 thru 0x44082a8 data_size 0xb(0x3569 bytes) 2024-09-12T07:20:50.507 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:50.508+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:50.509 INFO:teuthology.orchestra.run.smithi046.stdout:4422 trunc from 0xb5ad4e5 to 0xa88b1c4 2024-09-12T07:20:50.510 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:50.519 INFO:teuthology.orchestra.run.smithi046.stdout:4423 read 0x78ace1b thru 0x78b9810 (0xc9f6 bytes) 2024-09-12T07:20:50.519 INFO:teuthology.orchestra.run.smithi046.stdout:4424 clone 37 order 23 su 4194304 sc 9 2024-09-12T07:20:50.608 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:50.609+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:50.632 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126196 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.111426193, 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038184672000000003, 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126196 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6199999999999999e-07 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521351+0000", 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9520000000000001e-06 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521357+0000", 2024-09-12T07:20:50.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521362+0000", 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9180000000000002e-06 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559327+0000", 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037964880999999999 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559401+0000", 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3750000000000004e-05 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559533+0000", 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013262300000000001 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.637 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126334 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110845456, 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038163516000000001, 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126334 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521930+0000", 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521932+0000", 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9420000000000002e-06 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521938+0000", 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.127e-06 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521949+0000", 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1538e-05 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559814+0000", 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037864291000000001 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559903+0000", 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9409999999999999e-05 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560093+0000", 2024-09-12T07:20:50.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018984899999999999 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126336 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110817219000001, 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038384952, 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.642 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126336 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2e-07 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521962+0000", 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.591e-06 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521969+0000", 2024-09-12T07:20:50.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9199999999999998e-06 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521974+0000", 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2179999999999998e-06 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560137+0000", 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038163034999999998 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560193+0000", 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6425000000000003e-05 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560343+0000", 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000149343 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126337 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:50.646 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110794108, 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038587955, 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126337 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521983+0000", 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2079999999999999e-06 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521989+0000", 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8649999999999998e-06 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521994+0000", 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8720000000000001e-06 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560393+0000", 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038399232999999998 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560442+0000", 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8310999999999999e-05 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560569+0000", 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012708700000000001 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.650 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126339 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110774234000001, 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038809422000000003, 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126339 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.58e-07 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522003+0000", 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0059999999999999e-06 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522009+0000", 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522014+0000", 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0409999999999997e-06 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560605+0000", 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038590994000000003 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560650+0000", 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5336000000000001e-05 2024-09-12T07:20:50.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560810+0000", 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000159853 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126341 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110754905, 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039112779, 2024-09-12T07:20:50.655 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126341 2024-09-12T07:20:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:50.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6399999999999998e-07 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522022+0000", 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522028+0000", 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9089999999999997e-06 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522035+0000", 2024-09-12T07:20:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5200000000000003e-06 2024-09-12T07:20:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560857+0000", 2024-09-12T07:20:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038822239000000001 2024-09-12T07:20:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560955+0000", 2024-09-12T07:20:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7503999999999993e-05 2024-09-12T07:20:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561133+0000", 2024-09-12T07:20:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017827899999999999 2024-09-12T07:20:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.660 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126342 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:50.661 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110732969000001, 2024-09-12T07:20:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039296258000000001, 2024-09-12T07:20:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.662 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126342 2024-09-12T07:20:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.84e-07 2024-09-12T07:20:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522044+0000", 2024-09-12T07:20:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-12T07:20:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522057+0000", 2024-09-12T07:20:50.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330999999999999e-05 2024-09-12T07:20:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522062+0000", 2024-09-12T07:20:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3700000000000003e-06 2024-09-12T07:20:50.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561151+0000", 2024-09-12T07:20:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039089193000000001 2024-09-12T07:20:50.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561215+0000", 2024-09-12T07:20:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3305000000000005e-05 2024-09-12T07:20:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561338+0000", 2024-09-12T07:20:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012353900000000001 2024-09-12T07:20:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126343 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110704851000001, 2024-09-12T07:20:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039573481000000001, 2024-09-12T07:20:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.673 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126343 2024-09-12T07:20:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522071+0000", 2024-09-12T07:20:50.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2599999999999998e-07 2024-09-12T07:20:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522077+0000", 2024-09-12T07:20:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2049999999999999e-06 2024-09-12T07:20:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522084+0000", 2024-09-12T07:20:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1679999999999998e-06 2024-09-12T07:20:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522089+0000", 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561375+0000", 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039286692999999998 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561463+0000", 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7575000000000001e-05 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561644+0000", 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018070799999999999 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126344 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110678334999999, 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039759638, 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126344 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4900000000000001e-07 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522099+0000", 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4250000000000001e-06 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522106+0000", 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.455e-06 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522112+0000", 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3169999999999998e-06 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561676+0000", 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039564153999999997 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561728+0000", 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1848999999999998e-05 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561856+0000", 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012808900000000001 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126345 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110651742000002, 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040055773000000003, 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126345 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522124+0000", 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6600000000000002e-07 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522130+0000", 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.72e-06 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522140+0000", 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0217000000000001e-05 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522146+0000", 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3380000000000004e-06 2024-09-12T07:20:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561896+0000", 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039750343 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561997+0000", 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010129899999999999 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562179+0000", 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018149 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126346 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110444507, 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040058279000000002, 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126346 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522331+0000", 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522334+0000", 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.01e-06 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522346+0000", 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522353+0000", 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5590000000000002e-06 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562212+0000", 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039859232000000001 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562256+0000", 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4354999999999998e-05 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562389+0000", 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013237899999999999 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126348 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110411822, 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040352420999999999, 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126348 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522364+0000", 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.51e-07 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522366+0000", 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522373+0000", 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3339999999999996e-06 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522378+0000", 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8620000000000002e-06 2024-09-12T07:20:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562428+0000", 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040049840000000003 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562522+0000", 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3589000000000001e-05 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562716+0000", 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019395799999999999 2024-09-12T07:20:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126349 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1269760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110389013999999, 2024-09-12T07:20:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040535117000000002, 2024-09-12T07:20:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126349 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522389+0000", 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.317e-06 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522396+0000", 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522401+0000", 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5770000000000001e-06 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562754+0000", 2024-09-12T07:20:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040352271000000002 2024-09-12T07:20:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562809+0000", 2024-09-12T07:20:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5376e-05 2024-09-12T07:20:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562921+0000", 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112098 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126350 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110366525, 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040836117999999998, 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126350 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522409+0000", 2024-09-12T07:20:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9099999999999999e-07 2024-09-12T07:20:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522411+0000", 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1449999999999998e-06 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522417+0000", 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1070000000000001e-06 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522422+0000", 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7600000000000002e-06 2024-09-12T07:20:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562965+0000", 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040543467 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563058+0000", 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2905000000000006e-05 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563245+0000", 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186343 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126351 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110175684000001, 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040915973000000001, 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126351 2024-09-12T07:20:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522600+0000", 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7899999999999998e-07 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522604+0000", 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-06 2024-09-12T07:20:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522615+0000", 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1754e-05 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522622+0000", 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.798e-06 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563277+0000", 2024-09-12T07:20:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040654765000000002 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563340+0000", 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2580000000000001e-05 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563515+0000", 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017560699999999999 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126352 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110140390000002, 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041184365000000001, 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126352 2024-09-12T07:20:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.99e-07 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522637+0000", 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3180000000000001e-06 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522645+0000", 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0309999999999998e-06 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522651+0000", 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.789e-06 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563559+0000", 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040907261 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563634+0000", 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5708000000000001e-05 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563819+0000", 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000184659 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126353 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110114936999999, 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388632000000002, 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126353 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522661+0000", 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7299999999999996e-07 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522663+0000", 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-12T07:20:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522670+0000", 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6850000000000004e-06 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522675+0000", 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9139999999999999e-06 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563854+0000", 2024-09-12T07:20:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041178851000000002 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563912+0000", 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8584e-05 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564049+0000", 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000136427 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126354 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.110092411, 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041618080000000002, 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126354 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2399999999999999e-07 2024-09-12T07:20:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522685+0000", 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2469999999999998e-06 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522691+0000", 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.353e-06 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522697+0000", 2024-09-12T07:20:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.113e-06 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564085+0000", 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388104000000002 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564149+0000", 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4471e-05 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564301+0000", 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015146800000000001 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:129861 3.18 3:1cf16ce6:::rbd_data.116031710b3d.00000000000000ae:18 [read 40960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.860156700000001, 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038044815000000003, 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 129861 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772619+0000", 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5400000000000002e-07 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772620+0000", 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6980000000000001e-06 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772627+0000", 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3119999999999999e-06 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772631+0000", 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9010000000000002e-06 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810471+0000", 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037839308000000002 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810521+0000", 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9988999999999999e-05 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810663+0000", 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014225300000000001 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:129880 3.8 3:149a3dec:::rbd_data.116031710b3d.00000000000000ab:18 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.860116483999999, 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038109654999999999, 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 129880 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772659+0000", 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772661+0000", 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8279999999999999e-06 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772666+0000", 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6319999999999996e-06 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772672+0000", 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0510000000000002e-06 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810577+0000", 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037904895000000001 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810688+0000", 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011072400000000001 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810768+0000", 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0139e-05 2024-09-12T07:20:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:50.737 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:50.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:50.739 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:50.740+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:50.839 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:50.841+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:50.847 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:50.847 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:50.847 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:50.847 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:50.856 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:50.940 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:50.942+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:51.003 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:51.004 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:51.004 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:51.004 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:51.004 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:51.012 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:51.055 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:51.057+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:51.113 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:20:51.120 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:20:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 53.508183803000001, 2024-09-12T07:20:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:20:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:20:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:20:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:20:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:20:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:20:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:20:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:20:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:20:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:20:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 48.372723547, 2024-09-12T07:20:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:20:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:20:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:20:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:20:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:20:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:20:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:20:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:20:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:20:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:20:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:20:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:20:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:20:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 48.368496426, 2024-09-12T07:20:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:20:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:20:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:20:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:20:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:20:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:20:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:20:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:20:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:20:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:20:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:20:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.640745770999999, 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:20:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:20:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:20:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:20:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:20:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:20:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:20:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:20:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:20:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:20:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:20:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:20:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:20:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:20:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:20:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.551769182000001, 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:20:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:20:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:20:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:20:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:20:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:20:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:20:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:20:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:20:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:20:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:20:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68853 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'43, mlcod=43'39)", 2024-09-12T07:20:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.540776594999997, 2024-09-12T07:20:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017793166999999999, 2024-09-12T07:20:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580719+0000", 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4720000000000001e-06 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580732+0000", 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2926e-05 2024-09-12T07:20:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580746+0000", 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3291e-05 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580755+0000", 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6709999999999994e-06 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583766+0000", 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003010432 2024-09-12T07:20:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583794+0000", 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.811e-05 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598408+0000", 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014614472 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598511+0000", 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102793 2024-09-12T07:20:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.520217852000002, 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.517017316, 2024-09-12T07:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.513771568000003, 2024-09-12T07:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.511529737000004, 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:20:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:20:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:20:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:20:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:20:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:20:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:20:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:20:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69559 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'29, mlcod=41'27)", 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.437198688999999, 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018119258999999999, 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684297+0000", 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.192e-06 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691509+0000", 2024-09-12T07:20:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072122239999999997 2024-09-12T07:20:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691524+0000", 2024-09-12T07:20:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4562999999999999e-05 2024-09-12T07:20:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691533+0000", 2024-09-12T07:20:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1069999999999994e-06 2024-09-12T07:20:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691598+0000", 2024-09-12T07:20:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4666999999999996e-05 2024-09-12T07:20:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691620+0000", 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2092000000000001e-05 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702352+0000", 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010731785000000001 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702415+0000", 2024-09-12T07:20:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3628999999999997e-05 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69560 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'31, mlcod=41'27)", 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.429632566000002, 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016047555000000002, 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691863+0000", 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0299999999999999e-07 2024-09-12T07:20:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691897+0000", 2024-09-12T07:20:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4765999999999999e-05 2024-09-12T07:20:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691926+0000", 2024-09-12T07:20:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8413000000000001e-05 2024-09-12T07:20:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691934+0000", 2024-09-12T07:20:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3839999999999997e-06 2024-09-12T07:20:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694739+0000", 2024-09-12T07:20:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028043640000000002 2024-09-12T07:20:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694773+0000", 2024-09-12T07:20:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4514000000000002e-05 2024-09-12T07:20:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707854+0000", 2024-09-12T07:20:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013080591000000001 2024-09-12T07:20:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707910+0000", 2024-09-12T07:20:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.592e-05 2024-09-12T07:20:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "pg_update_log_missing(3.1a epoch 43/24 rep_tid 362 entries 43'32 (0'0) error 3:5ec4a676:::rbd_data.1160f022dd61.0000000000000019:head by client.4448.0:69639 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 41'27)", 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.429104852000002, 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015537618, 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692497+0000", 2024-09-12T07:20:51.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107154 2024-09-12T07:20:51.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692519+0000", 2024-09-12T07:20:51.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1043000000000002e-05 2024-09-12T07:20:51.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692542+0000", 2024-09-12T07:20:51.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3866000000000002e-05 2024-09-12T07:20:51.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702496+0000", 2024-09-12T07:20:51.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0099539169999999996 2024-09-12T07:20:51.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707927+0000", 2024-09-12T07:20:51.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005431038 2024-09-12T07:20:51.199 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.199 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.199 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.199 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:20:51.199 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:51.199 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.420422533, 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:51.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:20:51.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:20:51.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:20:51.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.416594445000001, 2024-09-12T07:20:51.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:20:51.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:20:51.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:20:51.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:20:51.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:20:51.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:20:51.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:20:51.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:20:51.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:20:51.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89144 3.a e53/24 3:54996542:::rbd_data.11607a56846.0000000000000001:head v 53'83, mlcod=53'81)", 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.428437690999999, 2024-09-12T07:20:51.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018298937000000001, 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693059+0000", 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.565e-06 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696238+0000", 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0031783269999999999 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696265+0000", 2024-09-12T07:20:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6982e-05 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696279+0000", 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4438e-05 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696366+0000", 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6978000000000006e-05 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696397+0000", 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0406999999999999e-05 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711297+0000", 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014900689999999999 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711356+0000", 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.855e-05 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.213 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89145 3.13 e53/24 3:ca7c3842:::rbd_data.11607a56846.0000000000000002:head v 53'202, mlcod=53'199)", 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.427406347999998, 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019835603, 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694091+0000", 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4990000000000001e-06 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697616+0000", 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035256749999999998 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697635+0000", 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8593999999999998e-05 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697646+0000", 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1436e-05 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697717+0000", 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0679000000000005e-05 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697748+0000", 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0723999999999999e-05 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713886+0000", 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016138550000000002 2024-09-12T07:20:51.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713924+0000", 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7446e-05 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 24.434681663999999, 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:20:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:20:51.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 24.422868019999999, 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:20:51.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:20:51.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 24.421531973, 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:20:51.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:51.231 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:51.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:51.233 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:51.233+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:51.241 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:51.259 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:51.259 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:51.274 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:51.291 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:51.291 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:51.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:51.308 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:51.324 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:51.334 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:51.334+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:51.341 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:51.358 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:51.358 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:51.383 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:51.383 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:51.389 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.2 to 1 (10000) 2024-09-12T07:20:51.392 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:51.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-12T07:20:51.436 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:51.438+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:51.510 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:51.550 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:51.552+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:51.559 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-12T07:20:51.636 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:51.636+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:51.657 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:51.658 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:51.666 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:51.737 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:51.737+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:51.773 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:51.773 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:51.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:51.818 INFO:teuthology.orchestra.run.smithi046.stdout:4425 trunc from 0xa88b1c4 to 0x7397d6 2024-09-12T07:20:51.838 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:51.838+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:51.840 INFO:teuthology.orchestra.run.smithi046.stdout:4426 read 0x2fa481 thru 0x306b35 (0xc6b5 bytes) 2024-09-12T07:20:51.856 INFO:teuthology.orchestra.run.smithi046.stdout:4429 writesame 0xa6ef95c thru 0xa6ff250 data_size 0x17(0xf8f5 bytes) 2024-09-12T07:20:51.866 INFO:teuthology.orchestra.run.smithi046.stdout:4430 write 0xe71ef89 thru 0xe7236bb (0x4733 bytes) 2024-09-12T07:20:51.870 INFO:teuthology.orchestra.run.smithi046.stdout:4433 punch from 0x69d498d to 0x69dc448, (0x7abb bytes) 2024-09-12T07:20:51.870 INFO:teuthology.orchestra.run.smithi046.stdout:4434 read 0x5c0bae3 thru 0x5c0d129 (0x1647 bytes) 2024-09-12T07:20:51.870 INFO:teuthology.orchestra.run.smithi046.stdout:4437 write 0x287a0bf thru 0x287e9d0 (0x4912 bytes) 2024-09-12T07:20:51.877 INFO:teuthology.orchestra.run.smithi046.stdout:4438 writesame 0xee590ce thru 0xee64984 data_size 0x1(0xb8b7 bytes) 2024-09-12T07:20:51.877 INFO:teuthology.orchestra.run.smithi046.stdout:4439 write 0x50934a thru 0x5132c0 (0x9f77 bytes) 2024-09-12T07:20:51.878 INFO:teuthology.orchestra.run.smithi046.stdout:4442 punch from 0x730e89e to 0x731e76f, (0xfed1 bytes) 2024-09-12T07:20:51.878 INFO:teuthology.orchestra.run.smithi046.stdout:4443 write 0xe8bdbfe thru 0xe8c1f9b (0x439e bytes) 2024-09-12T07:20:51.882 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:51.883 INFO:teuthology.orchestra.run.smithi046.stdout:4445 trunc from 0xee64985 to 0xda0bce0 2024-09-12T07:20:51.897 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:51.897 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:51.898 INFO:teuthology.orchestra.run.smithi046.stdout:4446 read 0x9c7041a thru 0x9c7a9b4 (0xa59b bytes) 2024-09-12T07:20:51.898 INFO:teuthology.orchestra.run.smithi046.stdout:4447 write 0x2162ad8 thru 0x2164895 (0x1dbe bytes) 2024-09-12T07:20:51.902 INFO:teuthology.orchestra.run.smithi046.stdout:4448 punch from 0xd39bfdb to 0xd3a267c, (0x66a1 bytes) 2024-09-12T07:20:51.902 INFO:teuthology.orchestra.run.smithi046.stdout:4449 read 0xa788bd7 thru 0xa79234e (0x9778 bytes) 2024-09-12T07:20:51.903 INFO:teuthology.orchestra.run.smithi046.stdout:4451 read 0xc27037c thru 0xc273795 (0x341a bytes) 2024-09-12T07:20:51.903 INFO:teuthology.orchestra.run.smithi046.stdout:4452 read 0xccd7ec0 thru 0xcce7bfc (0xfd3d bytes) 2024-09-12T07:20:51.903 INFO:teuthology.orchestra.run.smithi046.stdout:4453 punch from 0xc484783 to 0xc490c69, (0xc4e6 bytes) 2024-09-12T07:20:51.903 INFO:teuthology.orchestra.run.smithi046.stdout:4454 punch from 0x5a610d2 to 0x5a67fad, (0x6edb bytes) 2024-09-12T07:20:51.903 INFO:teuthology.orchestra.run.smithi046.stdout:4455 write 0x137e3aa thru 0x138e350 (0xffa7 bytes) 2024-09-12T07:20:51.905 INFO:teuthology.orchestra.run.smithi046.stdout:4458 read 0x6073557 thru 0x607ca77 (0x9521 bytes) 2024-09-12T07:20:51.906 INFO:teuthology.orchestra.run.smithi046.stdout:4461 write 0xafbd871 thru 0xafcab60 (0xd2f0 bytes) 2024-09-12T07:20:51.907 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:51.917 INFO:teuthology.orchestra.run.smithi046.stdout:4463 trunc from 0xda0bce0 to 0x63f77d1 2024-09-12T07:20:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:4466 write 0x2ae4b9a thru 0x2af1954 (0xcdbb bytes) 2024-09-12T07:20:51.941 INFO:teuthology.orchestra.run.smithi046.stdout:4467 trunc from 0x63f77d1 to 0xdf7c058 2024-09-12T07:20:51.941 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:51.941+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:51.949 INFO:teuthology.orchestra.run.smithi046.stdout:4468 trunc from 0xdf7c058 to 0x85191e1 2024-09-12T07:20:51.954 INFO:teuthology.orchestra.run.smithi046.stdout:4469 punch from 0x80494b to 0x80df91, (0x9646 bytes) 2024-09-12T07:20:51.955 INFO:teuthology.orchestra.run.smithi046.stdout:4471 read 0x7d4248f thru 0x7d4946f (0x6fe1 bytes) 2024-09-12T07:20:51.955 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:51.955 INFO:teuthology.orchestra.run.smithi046.stdout:4474 read 0x81924b7 thru 0x8192b36 (0x680 bytes) 2024-09-12T07:20:51.955 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:51.955 INFO:teuthology.orchestra.run.smithi046.stdout:4477 writesame 0x2a2fd2 thru 0x2b1ab9 data_size 0x4(0xeae8 bytes) 2024-09-12T07:20:51.963 INFO:teuthology.orchestra.run.smithi046.stdout:4478 read 0x7a9a79e thru 0x7aa74f1 (0xcd54 bytes) 2024-09-12T07:20:51.963 INFO:teuthology.orchestra.run.smithi046.stdout:4480 read 0x80558fc thru 0x8063588 (0xdc8d bytes) 2024-09-12T07:20:51.972 INFO:teuthology.orchestra.run.smithi046.stdout:4481 writesame 0xb28d0c4 thru 0xb298ecd data_size 0x46(0xbe0a bytes) 2024-09-12T07:20:51.972 INFO:teuthology.orchestra.run.smithi046.stdout:4482 read 0x4a036bb thru 0x4a03cf2 (0x638 bytes) 2024-09-12T07:20:51.973 INFO:teuthology.orchestra.run.smithi046.stdout:4483 trunc from 0xb298ece to 0x9ffc5cd 2024-09-12T07:20:51.978 INFO:teuthology.orchestra.run.smithi046.stdout:4484 read 0x471e1e1 thru 0x472dcaa (0xfaca bytes) 2024-09-12T07:20:51.979 INFO:teuthology.orchestra.run.smithi046.stdout:4485 punch from 0x639f6a5 to 0x63a02c9, (0xc24 bytes) 2024-09-12T07:20:51.979 INFO:teuthology.orchestra.run.smithi046.stdout:4487 read 0x5191393 thru 0x51921fa (0xe68 bytes) 2024-09-12T07:20:51.979 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:51.985 INFO:teuthology.orchestra.run.smithi046.stdout:4489 write 0xac57b01 thru 0xac62665 (0xab65 bytes) 2024-09-12T07:20:51.990 INFO:teuthology.orchestra.run.smithi046.stdout:4491 write 0x2b270ef thru 0x2b2aa5f (0x3971 bytes) 2024-09-12T07:20:51.991 INFO:teuthology.orchestra.run.smithi046.stdout:4492 write 0x39a8639 thru 0x39ac50e (0x3ed6 bytes) 2024-09-12T07:20:51.994 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.000 INFO:teuthology.orchestra.run.smithi046.stdout:4494 write 0xb9e74d8 thru 0xb9f6988 (0xf4b1 bytes) 2024-09-12T07:20:52.003 INFO:teuthology.orchestra.run.smithi046.stdout:4495 write 0x422600f thru 0x422d6e9 (0x76db bytes) 2024-09-12T07:20:52.007 INFO:teuthology.orchestra.run.smithi046.stdout:4497 punch from 0x277b53f to 0x2787b12, (0xc5d3 bytes) 2024-09-12T07:20:52.007 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.007 INFO:teuthology.orchestra.run.smithi046.stdout:4499 read 0x77619e2 thru 0x776c746 (0xad65 bytes) 2024-09-12T07:20:52.007 INFO:teuthology.orchestra.run.smithi046.stdout:4500 read 0x5bde29d thru 0x5beb63d (0xd3a1 bytes) 2024-09-12T07:20:52.007 INFO:teuthology.orchestra.run.smithi046.stdout:4502 write 0x9cff96f thru 0x9d029f5 (0x3087 bytes) 2024-09-12T07:20:52.017 INFO:teuthology.orchestra.run.smithi046.stdout:4504 writesame 0xc10273d thru 0xc1027d2 data_size 0x6(0x96 bytes) 2024-09-12T07:20:52.019 INFO:teuthology.orchestra.run.smithi046.stdout:4505 write 0x98f1aa2 thru 0x98fdc01 (0xc160 bytes) 2024-09-12T07:20:52.022 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:52.022 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:52.031 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:52.035 INFO:teuthology.orchestra.run.smithi046.stdout:4506 trunc from 0xc1027d3 to 0xd23764 2024-09-12T07:20:52.042 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:52.042+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:52.072 INFO:teuthology.orchestra.run.smithi046.stdout:4508 writesame 0x1b694b8 thru 0x1b781ad data_size 0x31(0xecf6 bytes) 2024-09-12T07:20:52.085 INFO:teuthology.orchestra.run.smithi046.stdout:4509 trunc from 0x1b781ae to 0xded33f3 2024-09-12T07:20:52.090 INFO:teuthology.orchestra.run.smithi046.stdout:4510 writesame 0x21a37c4 thru 0x21a6492 data_size 0x1(0x2ccf bytes) 2024-09-12T07:20:52.098 INFO:teuthology.orchestra.run.smithi046.stdout:4511 writesame 0xd60531d thru 0xd60e0e3 data_size 0x1(0x8dc7 bytes) 2024-09-12T07:20:52.118 INFO:teuthology.orchestra.run.smithi046.stdout:4512 read 0x267f2d1 thru 0x2680574 (0x12a4 bytes) 2024-09-12T07:20:52.119 INFO:teuthology.orchestra.run.smithi046.stdout:4513 read 0x466c339 thru 0x466d6e2 (0x13aa bytes) 2024-09-12T07:20:52.128 INFO:teuthology.orchestra.run.smithi046.stdout:4515 trunc from 0xded33f3 to 0x5b8f3b4 2024-09-12T07:20:52.137 INFO:teuthology.orchestra.run.smithi046.stdout:4518 read 0x4d8d1b3 thru 0x4d92adc (0x592a bytes) 2024-09-12T07:20:52.137 INFO:teuthology.orchestra.run.smithi046.stdout:4519 punch from 0x286753c to 0x28763ff, (0xeec3 bytes) 2024-09-12T07:20:52.142 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:52.144+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:52.145 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:52.146 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:52.146 INFO:teuthology.orchestra.run.smithi046.stdout:4520 writesame 0x835e793 thru 0x83682a2 data_size 0x18(0x9b10 bytes) 2024-09-12T07:20:52.150 INFO:teuthology.orchestra.run.smithi046.stdout:4521 trunc from 0x83682a3 to 0x46aad50 2024-09-12T07:20:52.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:52.157 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.174 INFO:teuthology.orchestra.run.smithi046.stdout:4524 writesame 0xde69f02 thru 0xde70d26 data_size 0xd(0x6e25 bytes) 2024-09-12T07:20:52.175 INFO:teuthology.orchestra.run.smithi046.stdout:4525 writesame 0x5db5c69 thru 0x5db7f3e data_size 0xe(0x22d6 bytes) 2024-09-12T07:20:52.185 INFO:teuthology.orchestra.run.smithi046.stdout:4527 writesame 0xe7758ae thru 0xe77c577 data_size 0x19(0x6cca bytes) 2024-09-12T07:20:52.185 INFO:teuthology.orchestra.run.smithi046.stdout:4529 write 0xda622cf thru 0xda691f8 (0x6f2a bytes) 2024-09-12T07:20:52.189 INFO:teuthology.orchestra.run.smithi046.stdout:4530 writesame 0x9b0d00e thru 0x9b108cf data_size 0x5(0x38c2 bytes) 2024-09-12T07:20:52.193 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.193 INFO:teuthology.orchestra.run.smithi046.stdout:4534 punch from 0xcd8613e to 0xcd87fbd, (0x1e7f bytes) 2024-09-12T07:20:52.193 INFO:teuthology.orchestra.run.smithi046.stdout:4535 write 0x1ceef38 thru 0x1cf410a (0x51d3 bytes) 2024-09-12T07:20:52.196 INFO:teuthology.orchestra.run.smithi046.stdout:4536 writesame 0x72c4e4c thru 0x72c8acc data_size 0x3(0x3c81 bytes) 2024-09-12T07:20:52.201 INFO:teuthology.orchestra.run.smithi046.stdout:4537 write 0x34d636f thru 0x34dca09 (0x669b bytes) 2024-09-12T07:20:52.205 INFO:teuthology.orchestra.run.smithi046.stdout:4538 writesame 0x224a674 thru 0x224e969 data_size 0x3(0x42f6 bytes) 2024-09-12T07:20:52.208 INFO:teuthology.orchestra.run.smithi046.stdout:4540 writesame 0xbbfda86 thru 0xbbfe4ce data_size 0x1(0xa49 bytes) 2024-09-12T07:20:52.212 INFO:teuthology.orchestra.run.smithi046.stdout:4541 read 0xa683355 thru 0xa685777 (0x2423 bytes) 2024-09-12T07:20:52.212 INFO:teuthology.orchestra.run.smithi046.stdout:4543 punch from 0x40ed428 to 0x40f43c6, (0x6f9e bytes) 2024-09-12T07:20:52.212 INFO:teuthology.orchestra.run.smithi046.stdout:4544 write 0x6fdd76f thru 0x6fe1cda (0x456c bytes) 2024-09-12T07:20:52.216 INFO:teuthology.orchestra.run.smithi046.stdout:4545 read 0xdff26c4 thru 0xdff6240 (0x3b7d bytes) 2024-09-12T07:20:52.216 INFO:teuthology.orchestra.run.smithi046.stdout:4546 write 0x3920de9 thru 0x392cd7c (0xbf94 bytes) 2024-09-12T07:20:52.219 INFO:teuthology.orchestra.run.smithi046.stdout:4547 writesame 0x2c074f2 thru 0x2c0ca1d data_size 0x5c(0x552c bytes) 2024-09-12T07:20:52.235 INFO:teuthology.orchestra.run.smithi046.stdout:4549 trunc from 0xe77c578 to 0x2b3c1e5 2024-09-12T07:20:52.243 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:52.244+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:52.263 INFO:teuthology.orchestra.run.smithi046.stdout:4551 write 0xba05af7 thru 0xba0d7f2 (0x7cfc bytes) 2024-09-12T07:20:52.267 INFO:teuthology.orchestra.run.smithi046.stdout:4552 write 0x7aceb30 thru 0x7ad0eec (0x23bd bytes) 2024-09-12T07:20:52.271 INFO:teuthology.orchestra.run.smithi046.stdout:4553 read 0x3c01b95 thru 0x3c1145f (0xf8cb bytes) 2024-09-12T07:20:52.271 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.271 INFO:teuthology.orchestra.run.smithi046.stdout:4556 punch from 0xaa45a2e to 0xaa4a386, (0x4958 bytes) 2024-09-12T07:20:52.271 INFO:teuthology.orchestra.run.smithi046.stdout:4557 read 0xab94271 thru 0xab94f90 (0xd20 bytes) 2024-09-12T07:20:52.281 INFO:teuthology.orchestra.run.smithi046.stdout:4558 trunc from 0xba0d7f3 to 0x239e015 2024-09-12T07:20:52.292 INFO:teuthology.orchestra.run.smithi046.stdout:4559 punch from 0x10f36ee to 0x10fba54, (0x8366 bytes) 2024-09-12T07:20:52.293 INFO:teuthology.orchestra.run.smithi046.stdout:4563 read 0x1cacaad thru 0x1cb81fe (0xb752 bytes) 2024-09-12T07:20:52.307 INFO:teuthology.orchestra.run.smithi046.stdout:4564 write 0x9ca5c32 thru 0x9cabf00 (0x62cf bytes) 2024-09-12T07:20:52.310 INFO:teuthology.orchestra.run.smithi046.stdout:4565 read 0x75dd6f thru 0x766741 (0x89d3 bytes) 2024-09-12T07:20:52.311 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:52.311 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:52.313 INFO:teuthology.orchestra.run.smithi046.stdout:4567 trunc from 0x9cabf01 to 0x7a86f14 2024-09-12T07:20:52.321 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:52.322 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.335 INFO:teuthology.orchestra.run.smithi046.stdout:4571 write 0xdde4be0 thru 0xddf0a6f (0xbe90 bytes) 2024-09-12T07:20:52.339 INFO:teuthology.orchestra.run.smithi046.stdout:4572 punch from 0x730e222 to 0x73188a0, (0xa67e bytes) 2024-09-12T07:20:52.339 INFO:teuthology.orchestra.run.smithi046.stdout:4573 read 0x9c717a6 thru 0x9c7e8df (0xd13a bytes) 2024-09-12T07:20:52.339 INFO:teuthology.orchestra.run.smithi046.stdout:4577 write 0x32bb4b7 thru 0x32bebc6 (0x3710 bytes) 2024-09-12T07:20:52.342 INFO:teuthology.orchestra.run.smithi046.stdout:4579 write 0xbedf3a2 thru 0xbee18e6 (0x2545 bytes) 2024-09-12T07:20:52.345 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.345 INFO:teuthology.orchestra.run.smithi046.stdout:4581 write 0xc7ef78a thru 0xc7f449e (0x4d15 bytes) 2024-09-12T07:20:52.349 INFO:teuthology.orchestra.run.smithi046.stdout:4582 writesame 0x3328b73 thru 0x332d3fe data_size 0x2(0x488c bytes) 2024-09-12T07:20:52.355 INFO:teuthology.orchestra.run.smithi046.stdout:4583 write 0xa0b8994 thru 0xa0bb691 (0x2cfe bytes) 2024-09-12T07:20:52.361 INFO:teuthology.orchestra.run.smithi046.stdout:4586 trunc from 0xddf0a70 to 0x9f0cb99 2024-09-12T07:20:52.365 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:52.367+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:52.376 INFO:teuthology.orchestra.run.smithi046.stdout:4587 write 0x7e09c65 thru 0x7e18f04 (0xf2a0 bytes) 2024-09-12T07:20:52.381 INFO:teuthology.orchestra.run.smithi046.stdout:4588 trunc from 0x9f0cb99 to 0x7385d48 2024-09-12T07:20:52.393 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.4 to 1 (10000) 2024-09-12T07:20:52.397 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.408 INFO:teuthology.orchestra.run.smithi046.stdout:4590 writesame 0x8e57e33 thru 0x8e65280 data_size 0x19(0xd44e bytes) 2024-09-12T07:20:52.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-12T07:20:52.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:20:52.413 INFO:teuthology.orchestra.run.smithi046.stdout:4591 trunc from 0x8e65281 to 0xcef927a 2024-09-12T07:20:52.418 INFO:teuthology.orchestra.run.smithi046.stdout:4592 read 0x54e7ef8 thru 0x54ee8b8 (0x69c1 bytes) 2024-09-12T07:20:52.419 INFO:teuthology.orchestra.run.smithi046.stdout:4593 punch from 0x620919c to 0x6214e12, (0xbc76 bytes) 2024-09-12T07:20:52.426 INFO:teuthology.orchestra.run.smithi046.stdout:4594 write 0xdec1992 thru 0xdecc1f6 (0xa865 bytes) 2024-09-12T07:20:52.431 INFO:teuthology.orchestra.run.smithi046.stdout:4595 write 0x6a09de8 thru 0x6a192e4 (0xf4fd bytes) 2024-09-12T07:20:52.437 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:52.437 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:52.441 INFO:teuthology.orchestra.run.smithi046.stdout:4597 write 0xe25794a thru 0xe25b314 (0x39cb bytes) 2024-09-12T07:20:52.445 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:52.445+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:52.445 INFO:teuthology.orchestra.run.smithi046.stdout:4598 write 0x6752865 thru 0x675383a (0xfd6 bytes) 2024-09-12T07:20:52.446 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:52.449 INFO:teuthology.orchestra.run.smithi046.stdout:4599 write 0xdf84715 thru 0xdf8d13b (0x8a27 bytes) 2024-09-12T07:20:52.451 INFO:teuthology.orchestra.run.smithi046.stdout:4600 writesame 0xab82710 thru 0xab83de1 data_size 0x17(0x16d2 bytes) 2024-09-12T07:20:52.455 INFO:teuthology.orchestra.run.smithi046.stdout:4601 write 0x3be95f9 thru 0x3bf5a7b (0xc483 bytes) 2024-09-12T07:20:52.457 INFO:teuthology.orchestra.run.smithi046.stdout:4602 write 0x5e98493 thru 0x5ea280d (0xa37b bytes) 2024-09-12T07:20:52.462 INFO:teuthology.orchestra.run.smithi046.stdout:4604 read 0x23f2e93 thru 0x23f3db6 (0xf24 bytes) 2024-09-12T07:20:52.463 INFO:teuthology.orchestra.run.smithi046.stdout:4605 write 0xa29ebbe thru 0xa2a33dc (0x481f bytes) 2024-09-12T07:20:52.466 INFO:teuthology.orchestra.run.smithi046.stdout:4606 read 0x902ab62 thru 0x9039762 (0xec01 bytes) 2024-09-12T07:20:52.479 INFO:teuthology.orchestra.run.smithi046.stdout:4608 trunc from 0xe25b315 to 0x45aec60 2024-09-12T07:20:52.493 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.493 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.501 INFO:teuthology.orchestra.run.smithi046.stdout:4615 writesame 0x50f22d9 thru 0x50f4f4e data_size 0x15(0x2c76 bytes) 2024-09-12T07:20:52.501 INFO:teuthology.orchestra.run.smithi046.stdout:4616 read 0x2e78594 thru 0x2e84b57 (0xc5c4 bytes) 2024-09-12T07:20:52.501 INFO:teuthology.orchestra.run.smithi046.stdout:4617 writesame 0x235f2b thru 0x2405bc data_size 0x8a(0xa692 bytes) 2024-09-12T07:20:52.504 INFO:teuthology.orchestra.run.smithi046.stdout:4618 writesame 0x8da894 thru 0x8df333 data_size 0x30(0x4aa0 bytes) 2024-09-12T07:20:52.507 INFO:teuthology.orchestra.run.smithi046.stdout:4619 punch from 0x4ab9acf to 0x4ac786e, (0xdd9f bytes) 2024-09-12T07:20:52.507 INFO:teuthology.orchestra.run.smithi046.stdout:4620 read 0x3c4c399 thru 0x3c4e4a3 (0x210b bytes) 2024-09-12T07:20:52.507 INFO:teuthology.orchestra.run.smithi046.stdout:4621 punch from 0x1e0fb8c to 0x1e149ba, (0x4e2e bytes) 2024-09-12T07:20:52.507 INFO:teuthology.orchestra.run.smithi046.stdout:4622 write 0x43f1887 thru 0x43f900e (0x7788 bytes) 2024-09-12T07:20:52.524 INFO:teuthology.orchestra.run.smithi046.stdout:4623 write 0xcdefcf4 thru 0xcdfde54 (0xe161 bytes) 2024-09-12T07:20:52.528 INFO:teuthology.orchestra.run.smithi046.stdout:4624 read 0x7aa258d thru 0x7ab182f (0xf2a3 bytes) 2024-09-12T07:20:52.528 INFO:teuthology.orchestra.run.smithi046.stdout:4625 write 0xcd44109 thru 0xcd4bd2c (0x7c24 bytes) 2024-09-12T07:20:52.529 INFO:teuthology.orchestra.run.smithi046.stdout:4627 read 0x86d28ef thru 0x86e00d2 (0xd7e4 bytes) 2024-09-12T07:20:52.529 INFO:teuthology.orchestra.run.smithi046.stdout:4628 read 0x1508b11 thru 0x15103a6 (0x7896 bytes) 2024-09-12T07:20:52.530 INFO:teuthology.orchestra.run.smithi046.stdout:4629 writesame 0xcbe007 thru 0xcc326b data_size 0x4f(0x5265 bytes) 2024-09-12T07:20:52.532 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.533 INFO:teuthology.orchestra.run.smithi046.stdout:4633 trunc from 0xcdfde55 to 0xbf90561 2024-09-12T07:20:52.541 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.541 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.541 INFO:teuthology.orchestra.run.smithi046.stdout:4638 read 0xbb74bc8 thru 0xbb84816 (0xfc4f bytes) 2024-09-12T07:20:52.541 INFO:teuthology.orchestra.run.smithi046.stdout:4640 read 0x42ac911 thru 0x42bb204 (0xe8f4 bytes) 2024-09-12T07:20:52.541 INFO:teuthology.orchestra.run.smithi046.stdout:4641 read 0x41bff1b thru 0x41ca805 (0xa8eb bytes) 2024-09-12T07:20:52.542 INFO:teuthology.orchestra.run.smithi046.stdout:4642 punch from 0x5ce2eb to 0x5d46f9, (0x640e bytes) 2024-09-12T07:20:52.542 INFO:teuthology.orchestra.run.smithi046.stdout:4647 read 0x4a52cfa thru 0x4a540c9 (0x13d0 bytes) 2024-09-12T07:20:52.542 INFO:teuthology.orchestra.run.smithi046.stdout:4648 trunc from 0xbf90561 to 0xc3bf952 2024-09-12T07:20:52.545 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:52.546+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:52.548 INFO:teuthology.orchestra.run.smithi046.stdout:4650 write 0x9e29079 thru 0x9e38fa0 (0xff28 bytes) 2024-09-12T07:20:52.552 INFO:teuthology.orchestra.run.smithi046.stdout:4651 write 0x85bde1c thru 0x85c2060 (0x4245 bytes) 2024-09-12T07:20:52.557 INFO:teuthology.orchestra.run.smithi046.stdout:4652 read 0x8ab45cc thru 0x8ab9bf2 (0x5627 bytes) 2024-09-12T07:20:52.557 INFO:teuthology.orchestra.run.smithi046.stdout:4653 read 0x9477f13 thru 0x94791e3 (0x12d1 bytes) 2024-09-12T07:20:52.557 INFO:teuthology.orchestra.run.smithi046.stdout:4655 read 0x17b7adb thru 0x17bc0b2 (0x45d8 bytes) 2024-09-12T07:20:52.557 INFO:teuthology.orchestra.run.smithi046.stdout:4656 writesame 0x60e39e3 thru 0x60eeaa1 data_size 0x1(0xb0bf bytes) 2024-09-12T07:20:52.574 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:52.574 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:52.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:52.594 INFO:teuthology.orchestra.run.smithi046.stdout:4657 write 0xc634c8a thru 0xc63c28b (0x7602 bytes) 2024-09-12T07:20:52.601 INFO:teuthology.orchestra.run.smithi046.stdout:4659 trunc from 0xc63c28c to 0xa390a3e 2024-09-12T07:20:52.612 INFO:teuthology.orchestra.run.smithi046.stdout:4660 read 0x841f7a thru 0x8424df (0x566 bytes) 2024-09-12T07:20:52.612 INFO:teuthology.orchestra.run.smithi046.stdout:4661 write 0x7697b02 thru 0x769a29c (0x279b bytes) 2024-09-12T07:20:52.617 INFO:teuthology.orchestra.run.smithi046.stdout:4662 write 0x4f07d5b thru 0x4f10a56 (0x8cfc bytes) 2024-09-12T07:20:52.621 INFO:teuthology.orchestra.run.smithi046.stdout:4663 write 0x5689b77 thru 0x5697997 (0xde21 bytes) 2024-09-12T07:20:52.624 INFO:teuthology.orchestra.run.smithi046.stdout:4664 punch from 0xfc7b87 to 0xfd72dd, (0xf756 bytes) 2024-09-12T07:20:52.624 INFO:teuthology.orchestra.run.smithi046.stdout:4666 write 0x5b7a998 thru 0x5b84d96 (0xa3ff bytes) 2024-09-12T07:20:52.627 INFO:teuthology.orchestra.run.smithi046.stdout:4667 write 0x9a21700 thru 0x9a23d24 (0x2625 bytes) 2024-09-12T07:20:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:4668 punch from 0x7325df8 to 0x7333d26, (0xdf2e bytes) 2024-09-12T07:20:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:4672 read 0x3a3e1ec thru 0x3a42503 (0x4318 bytes) 2024-09-12T07:20:52.631 INFO:teuthology.orchestra.run.smithi046.stdout:4673 write 0x40f67f2 thru 0x40f9bf8 (0x3407 bytes) 2024-09-12T07:20:52.633 INFO:teuthology.orchestra.run.smithi046.stdout:4674 punch from 0x53d667f to 0x53ddfeb, (0x796c bytes) 2024-09-12T07:20:52.633 INFO:teuthology.orchestra.run.smithi046.stdout:4675 read 0x7d355ff thru 0x7d35745 (0x147 bytes) 2024-09-12T07:20:52.633 INFO:teuthology.orchestra.run.smithi046.stdout:4676 writesame 0x4229b7c thru 0x422e0da data_size 0x2b(0x455f bytes) 2024-09-12T07:20:52.634 INFO:teuthology.orchestra.run.smithi046.stdout:4677 punch from 0x892bc2d to 0x892d56c, (0x193f bytes) 2024-09-12T07:20:52.635 INFO:teuthology.orchestra.run.smithi046.stdout:4678 read 0x6ecad57 thru 0x6ecbc59 (0xf03 bytes) 2024-09-12T07:20:52.635 INFO:teuthology.orchestra.run.smithi046.stdout:4679 read 0x864e7d thru 0x874652 (0xf7d6 bytes) 2024-09-12T07:20:52.635 INFO:teuthology.orchestra.run.smithi046.stdout:4680 writesame 0x3423eea thru 0x342d6e5 data_size 0x4(0x97fc bytes) 2024-09-12T07:20:52.641 INFO:teuthology.orchestra.run.smithi046.stdout:4682 writesame 0xcf5e22 thru 0xd03dc3 data_size 0x19(0xdfa2 bytes) 2024-09-12T07:20:52.643 INFO:teuthology.orchestra.run.smithi046.stdout:4683 writesame 0x7176032 thru 0x71794be data_size 0x1(0x348d bytes) 2024-09-12T07:20:52.646 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:52.648+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:52.651 INFO:teuthology.orchestra.run.smithi046.stdout:4684 writesame 0x3716593 thru 0x371b510 data_size 0x37(0x4f7e bytes) 2024-09-12T07:20:52.653 INFO:teuthology.orchestra.run.smithi046.stdout:4685 write 0x7b7ea71 thru 0x7b824c9 (0x3a59 bytes) 2024-09-12T07:20:52.661 INFO:teuthology.orchestra.run.smithi046.stdout:4686 write 0xa3f6b8d thru 0xa3fd067 (0x64db bytes) 2024-09-12T07:20:52.663 INFO:teuthology.orchestra.run.smithi046.stdout:4687 read 0x6171b04 thru 0x617e6ca (0xcbc7 bytes) 2024-09-12T07:20:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:4688 punch from 0x5a47edb to 0x5a56176, (0xe29b bytes) 2024-09-12T07:20:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:4689 read 0x227b082 thru 0x2282930 (0x78af bytes) 2024-09-12T07:20:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:4690 write 0x977c86d thru 0x9786818 (0x9fac bytes) 2024-09-12T07:20:52.675 INFO:teuthology.orchestra.run.smithi046.stdout:4691 write 0xbcc6cc4 thru 0xbccbe05 (0x5142 bytes) 2024-09-12T07:20:52.677 INFO:teuthology.orchestra.run.smithi046.stdout:4692 write 0x800a2a5 thru 0x800fa1f (0x577b bytes) 2024-09-12T07:20:52.682 INFO:teuthology.orchestra.run.smithi046.stdout:4694 clone 38 order 19 su 32768 sc 7 2024-09-12T07:20:52.714 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:52.714 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:52.723 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:52.748 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:52.749+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:52.848 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:52.849+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:52.870 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:52.871 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:52.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:52.950 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:52.951+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:53.028 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:53.028 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:53.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:53.079 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:53.080+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:53.144 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:53.144 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:53.151 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:53.153+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:53.152 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:53.252 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:53.252+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:53.300 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:53.300 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:53.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:53.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:53.393 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:53.393+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.5 to 1 (10000) 2024-09-12T07:20:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:20:53.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-12T07:20:53.454 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:53.455+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:53.468 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone37 from 0xbccbe06 (overlap 0x7397d6) to 0x0 2024-09-12T07:20:53.471 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:53.471 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:53.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:53.534 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-12T07:20:53.542 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:53.555 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:53.557+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:53.562 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:53.562 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:53.586 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:53.606 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:53.636 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:53.636 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:53.655 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:53.657+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:53.657 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:53.658 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:53.659 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:53.659 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:53.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:53.676 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:53.693 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:53.694 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:53.733 INFO:teuthology.orchestra.run.smithi046.stdout:4698 writesame 0x4f9f366 thru 0x4fa1b00 data_size 0x1(0x279b bytes) 2024-09-12T07:20:53.748 INFO:teuthology.orchestra.run.smithi046.stdout:4700 read 0x855c30 thru 0x856058 (0x429 bytes) 2024-09-12T07:20:53.748 INFO:teuthology.orchestra.run.smithi046.stdout:4702 writesame 0x90a9f34 thru 0x90aa6be data_size 0x1(0x78b bytes) 2024-09-12T07:20:53.756 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:53.757+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:53.761 INFO:teuthology.orchestra.run.smithi046.stdout:4704 trunc from 0xbccbe06 to 0x3b6fb07 2024-09-12T07:20:53.786 INFO:teuthology.orchestra.run.smithi046.stdout:4706 read 0x2a6cbca thru 0x2a75580 (0x89b7 bytes) 2024-09-12T07:20:53.798 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:53.798 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:53.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:53.798 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:53.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77440 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13156352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.552688029999999, 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051399724000000001, 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77440 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246267+0000", 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082256199999999999 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246271+0000", 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6700000000000002e-06 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246283+0000", 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.207e-05 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246291+0000", 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-12T07:20:53.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296645+0000", 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050353872000000001 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296729+0000", 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4215000000000006e-05 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296844+0000", 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114626 2024-09-12T07:20:53.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77446 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.551689515, 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051543643, 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77446 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246579+0000", 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013651399999999999 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246583+0000", 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1509999999999997e-06 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246593+0000", 2024-09-12T07:20:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5670000000000001e-06 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246599+0000", 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5189999999999998e-06 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297826+0000", 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051226381000000001 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297903+0000", 2024-09-12T07:20:53.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6710000000000002e-05 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297986+0000", 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3801000000000006e-05 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77447 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 5783552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:53.807 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.551521901000001, 2024-09-12T07:20:53.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051512637, 2024-09-12T07:20:53.808 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.808 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.808 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.808 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.808 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.808 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77447 2024-09-12T07:20:53.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.808 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246812+0000", 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000201506 2024-09-12T07:20:53.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246817+0000", 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246828+0000", 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1693e-05 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246836+0000", 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4429999999999997e-06 2024-09-12T07:20:53.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298021+0000", 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051184940999999998 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298067+0000", 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6292000000000003e-05 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298123+0000", 2024-09-12T07:20:53.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5828999999999997e-05 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77448 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.551282163000003, 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051506024999999997, 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77448 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246892+0000", 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1854999999999998e-05 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246895+0000", 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3129999999999999e-06 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246904+0000", 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6179999999999995e-06 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246911+0000", 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7939999999999997e-06 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298164+0000", 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051253559999999997 2024-09-12T07:20:53.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298249+0000", 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4549000000000007e-05 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298356+0000", 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107336 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77449 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.551210967000003, 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051582830000000003, 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77449 2024-09-12T07:20:53.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247137+0000", 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021599499999999999 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247142+0000", 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247154+0000", 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1878e-05 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247168+0000", 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4114000000000001e-05 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298391+0000", 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051223249999999998 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298446+0000", 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4366000000000001e-05 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298504+0000", 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8273000000000003e-05 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77450 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12681216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.550951396999999, 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051562375000000001, 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77450 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2799999999999996e-07 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247184+0000", 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247192+0000", 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5920000000000003e-06 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247197+0000", 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6720000000000001e-06 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298550+0000", 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051352082 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298641+0000", 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1236000000000005e-05 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298743+0000", 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102428 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.825 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77451 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7061504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.550927283, 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051813773, 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77451 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247335+0000", 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013012599999999999 2024-09-12T07:20:53.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247340+0000", 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.544e-06 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247351+0000", 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.111e-05 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247358+0000", 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9859999999999997e-06 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298909+0000", 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051551781999999997 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298960+0000", 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0594999999999999e-05 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299019+0000", 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8629999999999999e-05 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77452 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.550762503000001, 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051892134999999999, 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77452 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247535+0000", 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165508 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247552+0000", 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6514000000000001e-05 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247565+0000", 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3372e-05 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247577+0000", 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299064+0000", 2024-09-12T07:20:53.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051486697999999997 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299150+0000", 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6549000000000001e-05 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299262+0000", 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011135 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77453 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.550541782000003, 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051901623000000001, 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77453 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.835 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247710+0000", 2024-09-12T07:20:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011951 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247713+0000", 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4539999999999999e-06 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247720+0000", 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0339999999999999e-06 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247730+0000", 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0030000000000001e-05 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299297+0000", 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051566446000000002 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299382+0000", 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4963000000000006e-05 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299492+0000", 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011018600000000001 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77454 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7618560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.550393235000001, 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051921851999999998, 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77454 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247900+0000", 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016088699999999999 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247904+0000", 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6950000000000002e-06 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247912+0000", 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7129999999999994e-06 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247921+0000", 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0299999999999999e-06 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299526+0000", 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051604744000000001 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299601+0000", 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5129000000000005e-05 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299661+0000", 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9654000000000002e-05 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.843 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77455 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12746752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.550199366999998, 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051971254000000001, 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77455 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248181+0000", 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024790000000000001 2024-09-12T07:20:53.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248185+0000", 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5079999999999999e-06 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248197+0000", 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.132e-05 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248203+0000", 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6320000000000002e-06 2024-09-12T07:20:53.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299705+0000", 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051501342999999998 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299799+0000", 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4340999999999997e-05 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299904+0000", 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010521 2024-09-12T07:20:53.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77456 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7979008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.549917141999998, 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051860577999999997, 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77456 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248545+0000", 2024-09-12T07:20:53.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033009700000000001 2024-09-12T07:20:53.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248550+0000", 2024-09-12T07:20:53.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:20:53.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248560+0000", 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248568+0000", 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4939999999999997e-06 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299973+0000", 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051405470000000002 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300019+0000", 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5448999999999998e-05 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300076+0000", 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7073000000000001e-05 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77457 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13172736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:53.852 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.549550814, 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051737914000000003, 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77457 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248582+0000", 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248584+0000", 2024-09-12T07:20:53.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248593+0000", 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248609+0000", 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5664000000000001e-05 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300117+0000", 2024-09-12T07:20:53.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051507605999999997 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300212+0000", 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4919000000000004e-05 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300319+0000", 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010776399999999999 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77458 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.549508396, 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051863578, 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77458 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248627+0000", 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1389999999999999e-06 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248635+0000", 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4019999999999996e-06 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248641+0000", 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1689999999999997e-06 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300359+0000", 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051717889000000003 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300418+0000", 2024-09-12T07:20:53.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8689e-05 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300487+0000", 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9756000000000001e-05 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77459 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13238272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:53.861 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.549482703000002, 2024-09-12T07:20:53.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052076504000000003, 2024-09-12T07:20:53.862 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77459 2024-09-12T07:20:53.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.862 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250611+0000", 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019617879999999999 2024-09-12T07:20:53.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250618+0000", 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1829999999999997e-06 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250629+0000", 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0438000000000001e-05 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250648+0000", 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9355e-05 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300528+0000", 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049879252999999998 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300619+0000", 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1910000000000006e-05 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300726+0000", 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010657700000000001 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77467 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.547276883999999, 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05160754, 2024-09-12T07:20:53.866 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77467 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250856+0000", 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250858+0000", 2024-09-12T07:20:53.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9180000000000001e-06 2024-09-12T07:20:53.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250864+0000", 2024-09-12T07:20:53.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.066e-06 2024-09-12T07:20:53.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250869+0000", 2024-09-12T07:20:53.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-12T07:20:53.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302283+0000", 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051414591000000003 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302378+0000", 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4283000000000005e-05 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302463+0000", 2024-09-12T07:20:53.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5249999999999999e-05 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77468 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14909440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.547257311000003, 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051742162000000001, 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77468 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250877+0000", 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.815e-06 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250883+0000", 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250888+0000", 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302496+0000", 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051607394000000001 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302553+0000", 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6971000000000001e-05 2024-09-12T07:20:53.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302617+0000", 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4418999999999994e-05 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77469 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14974976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.547237131999999, 2024-09-12T07:20:53.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965694, 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77469 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:53.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:20:53.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:20:53.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250897+0000", 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9130000000000001e-06 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250903+0000", 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0129999999999999e-06 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250908+0000", 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9679999999999997e-06 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302661+0000", 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051753066 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302755+0000", 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3376999999999997e-05 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302861+0000", 2024-09-12T07:20:53.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105924 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77470 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12828672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.547217455000002, 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052098589000000001, 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.880 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77470 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:53.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250917+0000", 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8190000000000001e-06 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250923+0000", 2024-09-12T07:20:53.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250928+0000", 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9420000000000003e-06 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302894+0000", 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965863000000001 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302955+0000", 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1235999999999994e-05 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303013+0000", 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8167000000000001e-05 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77471 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12894208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:20:53.884 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.547197908999998, 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052326263999999997, 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77471 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:53.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256961+0000", 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060271209999999999 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256972+0000", 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0755e-05 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256986+0000", 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845e-05 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256996+0000", 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303060+0000", 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.046063661999999998 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303149+0000", 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8949999999999994e-05 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303261+0000", 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011195 2024-09-12T07:20:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:53.889 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:53.890 INFO:teuthology.orchestra.run.smithi046.stdout:4707 trunc from 0x3b6fb07 to 0xe9b83cb 2024-09-12T07:20:53.890 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:53.890 INFO:teuthology.orchestra.run.smithi046.stdout:4710 write 0x8ab89e2 thru 0x8abed6e (0x638d bytes) 2024-09-12T07:20:53.891 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:53.893 INFO:teuthology.orchestra.run.smithi046.stdout:4712 write 0x81ff1ec thru 0x8207885 (0x869a bytes) 2024-09-12T07:20:53.893 INFO:teuthology.orchestra.run.smithi046.stdout:4713 read 0xcf03508 thru 0xcf07003 (0x3afc bytes) 2024-09-12T07:20:53.893 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:53.893 INFO:teuthology.orchestra.run.smithi046.stdout:4715 write 0xb629ac1 thru 0xb631789 (0x7cc9 bytes) 2024-09-12T07:20:53.893 INFO:teuthology.orchestra.run.smithi046.stdout:4716 read 0x608fa01 thru 0x609b177 (0xb777 bytes) 2024-09-12T07:20:53.893 INFO:teuthology.orchestra.run.smithi046.stdout:4717 write 0x5caa4ab thru 0x5cb4425 (0x9f7b bytes) 2024-09-12T07:20:53.893 INFO:teuthology.orchestra.run.smithi046.stdout:4718 write 0x3670bbf thru 0x3675595 (0x49d7 bytes) 2024-09-12T07:20:53.894 INFO:teuthology.orchestra.run.smithi046.stdout:4719 punch from 0x72fe3a0 to 0x7301550, (0x31b0 bytes) 2024-09-12T07:20:53.894 INFO:teuthology.orchestra.run.smithi046.stdout:4720 writesame 0x9730760 thru 0x97395c5 data_size 0xb(0x8e66 bytes) 2024-09-12T07:20:53.894 INFO:teuthology.orchestra.run.smithi046.stdout:4723 trunc from 0xe9b83cb to 0xce7a23f 2024-09-12T07:20:53.894 INFO:teuthology.orchestra.run.smithi046.stdout:4724 writesame 0x8ef16c7 thru 0x8efcc45 data_size 0x1(0xb57f bytes) 2024-09-12T07:20:53.894 INFO:teuthology.orchestra.run.smithi046.stdout:4725 read 0x14a6499 thru 0x14aa509 (0x4071 bytes) 2024-09-12T07:20:53.894 INFO:teuthology.orchestra.run.smithi046.stdout:4726 trunc from 0xce7a23f to 0x69cbf5d 2024-09-12T07:20:53.894 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:53.894 INFO:teuthology.orchestra.run.smithi046.stdout:4728 read 0x22049c7 thru 0x220adac (0x63e6 bytes) 2024-09-12T07:20:53.894 INFO:teuthology.orchestra.run.smithi046.stdout:4729 write 0x4da27ae thru 0x4daf0ec (0xc93f bytes) 2024-09-12T07:20:53.894 INFO:teuthology.orchestra.run.smithi046.stdout:4731 punch from 0x7fcbfc to 0x80618d, (0x9591 bytes) 2024-09-12T07:20:53.894 INFO:teuthology.orchestra.run.smithi046.stdout:4732 writesame 0x1ad636f thru 0x1adafdd data_size 0x1(0x4c6f bytes) 2024-09-12T07:20:53.895 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:53.891+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:53.898 INFO:teuthology.orchestra.run.smithi046.stdout:4734 trunc from 0x69cbf5d to 0x7fe4f5c 2024-09-12T07:20:53.917 INFO:teuthology.orchestra.run.smithi046.stdout:4735 write 0xe98bdca thru 0xe99ac47 (0xee7e bytes) 2024-09-12T07:20:53.926 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:53.926 INFO:teuthology.orchestra.run.smithi046.stdout:4740 read 0x3467d2f thru 0x3469ee9 (0x21bb bytes) 2024-09-12T07:20:53.940 INFO:teuthology.orchestra.run.smithi046.stdout:4741 trunc from 0xe99ac48 to 0x23ed650 2024-09-12T07:20:53.971 INFO:teuthology.orchestra.run.smithi046.stdout:4742 read 0xc40b72 thru 0xc46b4d (0x5fdc bytes) 2024-09-12T07:20:53.981 INFO:teuthology.orchestra.run.smithi046.stdout:4745 write 0x5e5d861 thru 0x5e659dc (0x817c bytes) 2024-09-12T07:20:53.989 INFO:teuthology.orchestra.run.smithi046.stdout:4747 punch from 0x5e1fd94 to 0x5e25a91, (0x5cfd bytes) 2024-09-12T07:20:53.989 INFO:teuthology.orchestra.run.smithi046.stdout:4748 read 0x3a1ac0 thru 0x3aff3f (0xe480 bytes) 2024-09-12T07:20:53.989 INFO:teuthology.orchestra.run.smithi046.stdout:4750 read 0x2b008ad thru 0x2b01e52 (0x15a6 bytes) 2024-09-12T07:20:53.989 INFO:teuthology.orchestra.run.smithi046.stdout:4751 read 0x5daaaf5 thru 0x5db9d36 (0xf242 bytes) 2024-09-12T07:20:53.989 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:53.989 INFO:teuthology.orchestra.run.smithi046.stdout:4754 read 0x18edf57 thru 0x18fad89 (0xce33 bytes) 2024-09-12T07:20:53.989 INFO:teuthology.orchestra.run.smithi046.stdout:4757 writesame 0x42a6185 thru 0x42a9d04 data_size 0x44(0x3b80 bytes) 2024-09-12T07:20:53.990 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:53.991+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:53.990 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:54.005 INFO:teuthology.orchestra.run.smithi046.stdout:4759 writesame 0xe4c3ccd thru 0xe4cfa4d data_size 0x67(0xbd81 bytes) 2024-09-12T07:20:54.020 INFO:teuthology.orchestra.run.smithi046.stdout:4761 trunc from 0xe4cfa4e to 0xf3ed15 2024-09-12T07:20:54.022 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:54.022 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:20:54.023 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:54.035 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:54.054 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:54.054 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:54.054 INFO:teuthology.orchestra.run.smithi046.stdout:4764 read 0x111639 thru 0x119fc1 (0x8989 bytes) 2024-09-12T07:20:54.055 INFO:teuthology.orchestra.run.smithi046.stdout:4768 punch from 0x605201 to 0x608fb0, (0x3daf bytes) 2024-09-12T07:20:54.066 INFO:teuthology.orchestra.run.smithi046.stdout:4769 write 0x598e694 thru 0x5992407 (0x3d74 bytes) 2024-09-12T07:20:54.069 INFO:teuthology.orchestra.run.smithi046.stdout:4771 read 0x2eb5033 thru 0x2ebcd05 (0x7cd3 bytes) 2024-09-12T07:20:54.081 INFO:teuthology.orchestra.run.smithi046.stdout:4772 write 0xad50906 thru 0xad522b0 (0x19ab bytes) 2024-09-12T07:20:54.084 INFO:teuthology.orchestra.run.smithi046.stdout:4773 write 0x4715801 thru 0x47188e4 (0x30e4 bytes) 2024-09-12T07:20:54.087 INFO:teuthology.orchestra.run.smithi046.stdout:4774 write 0x4c8c1e4 thru 0x4c91707 (0x5524 bytes) 2024-09-12T07:20:54.091 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:54.093+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:54.099 INFO:teuthology.orchestra.run.smithi046.stdout:4775 write 0xccb7494 thru 0xccc206f (0xabdc bytes) 2024-09-12T07:20:54.109 INFO:teuthology.orchestra.run.smithi046.stdout:4776 read 0x71e8689 thru 0x71e8a03 (0x37b bytes) 2024-09-12T07:20:54.109 INFO:teuthology.orchestra.run.smithi046.stdout:4777 read 0x1272355 thru 0x127a519 (0x81c5 bytes) 2024-09-12T07:20:54.109 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:54.109 INFO:teuthology.orchestra.run.smithi046.stdout:4779 read 0x66eceee thru 0x66f94e5 (0xc5f8 bytes) 2024-09-12T07:20:54.109 INFO:teuthology.orchestra.run.smithi046.stdout:4780 trunc from 0xccc2070 to 0xd7c8fd0 2024-09-12T07:20:54.117 INFO:teuthology.orchestra.run.smithi046.stdout:4781 writesame 0x34b93b6 thru 0x34c222d data_size 0x2f(0x8e78 bytes) 2024-09-12T07:20:54.122 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:54.122 INFO:teuthology.orchestra.run.smithi046.stdout:4784 clone 39 order 23 su 32768 sc 4 2024-09-12T07:20:54.168 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:20:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:20:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:20:54.168 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:54.177 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:54.218 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:54.218+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:54.294 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:54.296+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:54.325 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:54.325 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:54.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:54.325 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:54.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.325 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126196 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.325 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.804782967000001, 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038184672000000003, 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126196 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:54.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6199999999999999e-07 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521351+0000", 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9520000000000001e-06 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521357+0000", 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521362+0000", 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9180000000000002e-06 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559327+0000", 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037964880999999999 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559401+0000", 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3750000000000004e-05 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559533+0000", 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013262300000000001 2024-09-12T07:20:54.329 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126334 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.804202230000001, 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038163516000000001, 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126334 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521930+0000", 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521932+0000", 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9420000000000002e-06 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521938+0000", 2024-09-12T07:20:54.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.127e-06 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521949+0000", 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1538e-05 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559814+0000", 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037864291000000001 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559903+0000", 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9409999999999999e-05 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560093+0000", 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018984899999999999 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126336 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:54.334 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.804173992999999, 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038384952, 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126336 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2e-07 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521962+0000", 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.591e-06 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521969+0000", 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9199999999999998e-06 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521974+0000", 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2179999999999998e-06 2024-09-12T07:20:54.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560137+0000", 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038163034999999998 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560193+0000", 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6425000000000003e-05 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560343+0000", 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000149343 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126337 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.804150881999998, 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038587955, 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.339 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126337 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521983+0000", 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2079999999999999e-06 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521989+0000", 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8649999999999998e-06 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521994+0000", 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8720000000000001e-06 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560393+0000", 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038399232999999998 2024-09-12T07:20:54.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560442+0000", 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8310999999999999e-05 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560569+0000", 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012708700000000001 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126339 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.804131007999999, 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038809422000000003, 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126339 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.344 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.58e-07 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522003+0000", 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0059999999999999e-06 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522009+0000", 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522014+0000", 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0409999999999997e-06 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560605+0000", 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038590994000000003 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560650+0000", 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5336000000000001e-05 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560810+0000", 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000159853 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126341 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.804111679000002, 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039112779, 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126341 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6399999999999998e-07 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522022+0000", 2024-09-12T07:20:54.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522028+0000", 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9089999999999997e-06 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522035+0000", 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5200000000000003e-06 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560857+0000", 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038822239000000001 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560955+0000", 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7503999999999993e-05 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561133+0000", 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017827899999999999 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.352 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126342 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.804089742999999, 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039296258000000001, 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.353 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126342 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.84e-07 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522044+0000", 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522057+0000", 2024-09-12T07:20:54.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330999999999999e-05 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522062+0000", 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3700000000000003e-06 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561151+0000", 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039089193000000001 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561215+0000", 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3305000000000005e-05 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561338+0000", 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012353900000000001 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126343 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.804061624999999, 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039573481000000001, 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126343 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522071+0000", 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2599999999999998e-07 2024-09-12T07:20:54.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522077+0000", 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2049999999999999e-06 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522084+0000", 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1679999999999998e-06 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522089+0000", 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561375+0000", 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039286692999999998 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561463+0000", 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7575000000000001e-05 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561644+0000", 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018070799999999999 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126344 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.804035109000001, 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039759638, 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126344 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4900000000000001e-07 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522099+0000", 2024-09-12T07:20:54.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4250000000000001e-06 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522106+0000", 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.455e-06 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522112+0000", 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3169999999999998e-06 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561676+0000", 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039564153999999997 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561728+0000", 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1848999999999998e-05 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561856+0000", 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012808900000000001 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126345 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.804008516, 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040055773000000003, 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126345 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522124+0000", 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6600000000000002e-07 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522130+0000", 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.72e-06 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522140+0000", 2024-09-12T07:20:54.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0217000000000001e-05 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522146+0000", 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3380000000000004e-06 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561896+0000", 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039750343 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561997+0000", 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010129899999999999 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562179+0000", 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018149 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126346 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.371 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.803801280999998, 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040058279000000002, 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126346 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522331+0000", 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522334+0000", 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.01e-06 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522346+0000", 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522353+0000", 2024-09-12T07:20:54.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5590000000000002e-06 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562212+0000", 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039859232000000001 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562256+0000", 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4354999999999998e-05 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562389+0000", 2024-09-12T07:20:54.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013237899999999999 2024-09-12T07:20:54.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.376 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.376 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126348 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.376 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:54.376 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.803768596000001, 2024-09-12T07:20:54.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040352420999999999, 2024-09-12T07:20:54.376 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126348 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522364+0000", 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.51e-07 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522366+0000", 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522373+0000", 2024-09-12T07:20:54.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3339999999999996e-06 2024-09-12T07:20:54.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522378+0000", 2024-09-12T07:20:54.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8620000000000002e-06 2024-09-12T07:20:54.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562428+0000", 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040049840000000003 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562522+0000", 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3589000000000001e-05 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562716+0000", 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019395799999999999 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126349 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1269760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.803745788000001, 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040535117000000002, 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126349 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:54.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:20:54.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522389+0000", 2024-09-12T07:20:54.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.317e-06 2024-09-12T07:20:54.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522396+0000", 2024-09-12T07:20:54.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522401+0000", 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5770000000000001e-06 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562754+0000", 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040352271000000002 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562809+0000", 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5376e-05 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562921+0000", 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112098 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126350 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:54.385 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.803723299000001, 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040836117999999998, 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126350 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:54.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522409+0000", 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9099999999999999e-07 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522411+0000", 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1449999999999998e-06 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522417+0000", 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1070000000000001e-06 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522422+0000", 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7600000000000002e-06 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562965+0000", 2024-09-12T07:20:54.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040543467 2024-09-12T07:20:54.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563058+0000", 2024-09-12T07:20:54.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2905000000000006e-05 2024-09-12T07:20:54.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563245+0000", 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186343 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126351 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.803532457999999, 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040915973000000001, 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.390 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126351 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522600+0000", 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7899999999999998e-07 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522604+0000", 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-06 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522615+0000", 2024-09-12T07:20:54.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1754e-05 2024-09-12T07:20:54.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522622+0000", 2024-09-12T07:20:54.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.798e-06 2024-09-12T07:20:54.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563277+0000", 2024-09-12T07:20:54.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040654765000000002 2024-09-12T07:20:54.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563340+0000", 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2580000000000001e-05 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563515+0000", 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017560699999999999 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.397 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126352 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.803497163999999, 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041184365000000001, 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126352 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:54.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.99e-07 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522637+0000", 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3180000000000001e-06 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522645+0000", 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0309999999999998e-06 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522651+0000", 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.789e-06 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563559+0000", 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040907261 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563634+0000", 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5708000000000001e-05 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563819+0000", 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000184659 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126353 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.803471711, 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388632000000002, 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.402 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126353 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522661+0000", 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7299999999999996e-07 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522663+0000", 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-12T07:20:54.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522670+0000", 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6850000000000004e-06 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522675+0000", 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9139999999999999e-06 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563854+0000", 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041178851000000002 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563912+0000", 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8584e-05 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564049+0000", 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000136427 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.406 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126354 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.803449185000002, 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041618080000000002, 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126354 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2399999999999999e-07 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522685+0000", 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2469999999999998e-06 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522691+0000", 2024-09-12T07:20:54.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.353e-06 2024-09-12T07:20:54.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522697+0000", 2024-09-12T07:20:54.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.113e-06 2024-09-12T07:20:54.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564085+0000", 2024-09-12T07:20:54.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388104000000002 2024-09-12T07:20:54.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564149+0000", 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4471e-05 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564301+0000", 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015146800000000001 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.411 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:129861 3.18 3:1cf16ce6:::rbd_data.116031710b3d.00000000000000ae:18 [read 40960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.553513473999999, 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038044815000000003, 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 129861 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772619+0000", 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5400000000000002e-07 2024-09-12T07:20:54.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772620+0000", 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6980000000000001e-06 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772627+0000", 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3119999999999999e-06 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772631+0000", 2024-09-12T07:20:54.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9010000000000002e-06 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810471+0000", 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037839308000000002 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810521+0000", 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9988999999999999e-05 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810663+0000", 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014225300000000001 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:129880 3.8 3:149a3dec:::rbd_data.116031710b3d.00000000000000ab:18 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.553473258, 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038109654999999999, 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:54.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 129880 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772659+0000", 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772661+0000", 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8279999999999999e-06 2024-09-12T07:20:54.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:54.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772666+0000", 2024-09-12T07:20:54.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6319999999999996e-06 2024-09-12T07:20:54.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772672+0000", 2024-09-12T07:20:54.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0510000000000002e-06 2024-09-12T07:20:54.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:54.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810577+0000", 2024-09-12T07:20:54.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037904895000000001 2024-09-12T07:20:54.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:54.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810688+0000", 2024-09-12T07:20:54.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011072400000000001 2024-09-12T07:20:54.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:54.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:54.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:54.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810768+0000", 2024-09-12T07:20:54.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0139e-05 2024-09-12T07:20:54.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.421 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:54.421 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:54.421 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:54.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.6 to 1 (10000) 2024-09-12T07:20:54.424 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:54.423+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:54.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-12T07:20:54.471 INFO:teuthology.orchestra.run.smithi046.stdout:leaving image image_client.0-clone38 intact 2024-09-12T07:20:54.523 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:54.523+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:54.534 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:54.534 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:54.534 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:54.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:54.624 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:54.624+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:54.625 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-12T07:20:54.636 INFO:teuthology.orchestra.run.smithi046.stdout:4786 trunc from 0xd7c8fd0 to 0x42c50b5 2024-09-12T07:20:54.655 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:54.655 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:54.655 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:54.656 INFO:teuthology.orchestra.run.smithi046.stdout:4788 write 0xec362d thru 0xecf93b (0xc30f bytes) 2024-09-12T07:20:54.666 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:54.714 INFO:teuthology.orchestra.run.smithi046.stdout:4789 write 0xe663315 thru 0xe670652 (0xd33e bytes) 2024-09-12T07:20:54.723 INFO:teuthology.orchestra.run.smithi046.stdout:4790 read 0x794bbf thru 0x7a44c5 (0xf907 bytes) 2024-09-12T07:20:54.724 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:54.726+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:54.735 INFO:teuthology.orchestra.run.smithi046.stdout:4792 trunc from 0xe670653 to 0x433cf0b 2024-09-12T07:20:54.755 INFO:teuthology.orchestra.run.smithi046.stdout:4793 writesame 0x71632a8 thru 0x716afcc data_size 0x3(0x7d25 bytes) 2024-09-12T07:20:54.767 INFO:teuthology.orchestra.run.smithi046.stdout:4794 writesame 0xcc3d7bd thru 0xcc45579 data_size 0x1(0x7dbd bytes) 2024-09-12T07:20:54.767 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:54.780 INFO:teuthology.orchestra.run.smithi046.stdout:4797 trunc from 0xcc4557a to 0x14582b6 2024-09-12T07:20:54.787 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:20:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 57.173836110000003, 2024-09-12T07:20:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:20:54.787 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:20:54.791 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 52.038375854000002, 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:20:54.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:20:54.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 52.034148733000002, 2024-09-12T07:20:54.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:20:54.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.800 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.801 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:20:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.306398078000001, 2024-09-12T07:20:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:20:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.804 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:20:54.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.217421489000003, 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:20:54.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68853 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'43, mlcod=43'39)", 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.206428901999999, 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017793166999999999, 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580719+0000", 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4720000000000001e-06 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580732+0000", 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2926e-05 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580746+0000", 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3291e-05 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580755+0000", 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6709999999999994e-06 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583766+0000", 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003010432 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583794+0000", 2024-09-12T07:20:54.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.811e-05 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598408+0000", 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014614472 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598511+0000", 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102793 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.185870158999997, 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:20:54.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.182669623000002, 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:20:54.822 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:20:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:20:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:20:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:20:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:20:54.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.179423874999998, 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:54.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.177182043999998, 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:20:54.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:20:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69559 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'29, mlcod=41'27)", 2024-09-12T07:20:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:54.836 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.102850996000001, 2024-09-12T07:20:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018119258999999999, 2024-09-12T07:20:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684297+0000", 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.192e-06 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691509+0000", 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072122239999999997 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691524+0000", 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4562999999999999e-05 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691533+0000", 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1069999999999994e-06 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691598+0000", 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4666999999999996e-05 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691620+0000", 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2092000000000001e-05 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702352+0000", 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010731785000000001 2024-09-12T07:20:54.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702415+0000", 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3628999999999997e-05 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69560 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'31, mlcod=41'27)", 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.095284872999997, 2024-09-12T07:20:54.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016047555000000002, 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691863+0000", 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0299999999999999e-07 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691897+0000", 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4765999999999999e-05 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691926+0000", 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8413000000000001e-05 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691934+0000", 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3839999999999997e-06 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694739+0000", 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028043640000000002 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694773+0000", 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4514000000000002e-05 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707854+0000", 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013080591000000001 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707910+0000", 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.592e-05 2024-09-12T07:20:54.845 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "pg_update_log_missing(3.1a epoch 43/24 rep_tid 362 entries 43'32 (0'0) error 3:5ec4a676:::rbd_data.1160f022dd61.0000000000000019:head by client.4448.0:69639 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 41'27)", 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.094757158999997, 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015537618, 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692497+0000", 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107154 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692519+0000", 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1043000000000002e-05 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692542+0000", 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3866000000000002e-05 2024-09-12T07:20:54.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702496+0000", 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0099539169999999996 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707927+0000", 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005431038 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.086074840000002, 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:20:54.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:20:54.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.082246752000003, 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.854 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:20:54.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89144 3.a e53/24 3:54996542:::rbd_data.11607a56846.0000000000000001:head v 53'83, mlcod=53'81)", 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.094089998000001, 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018298937000000001, 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693059+0000", 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.565e-06 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696238+0000", 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0031783269999999999 2024-09-12T07:20:54.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696265+0000", 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6982e-05 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696279+0000", 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4438e-05 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696366+0000", 2024-09-12T07:20:54.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6978000000000006e-05 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696397+0000", 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0406999999999999e-05 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711297+0000", 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014900689999999999 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711356+0000", 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.855e-05 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89145 3.13 e53/24 3:ca7c3842:::rbd_data.11607a56846.0000000000000002:head v 53'202, mlcod=53'199)", 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.093058655, 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019835603, 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.863 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694091+0000", 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4990000000000001e-06 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697616+0000", 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035256749999999998 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697635+0000", 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8593999999999998e-05 2024-09-12T07:20:54.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697646+0000", 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1436e-05 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697717+0000", 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0679000000000005e-05 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697748+0000", 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0723999999999999e-05 2024-09-12T07:20:54.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713886+0000", 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016138550000000002 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713924+0000", 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7446e-05 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.100333971000001, 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:20:54.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.088520327000001, 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.872 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 28.087184279999999, 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:20:54.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:20:54.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:54.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:20:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:20:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:54.881 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:54.881 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:54.883 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:54.884 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:54.884+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:54.884 INFO:teuthology.orchestra.run.smithi046.stdout:4798 punch from 0x11b9735 to 0x11bc412, (0x2cdd bytes) 2024-09-12T07:20:54.885 INFO:teuthology.orchestra.run.smithi046.stdout:4799 write 0x6fba116 thru 0x6fc0610 (0x64fb bytes) 2024-09-12T07:20:54.885 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:54.885 INFO:teuthology.orchestra.run.smithi046.stdout:4805 write 0x5afae49 thru 0x5afe305 (0x34bd bytes) 2024-09-12T07:20:54.885 INFO:teuthology.orchestra.run.smithi046.stdout:4806 write 0x17ba02f thru 0x17c10a2 (0x7074 bytes) 2024-09-12T07:20:54.885 INFO:teuthology.orchestra.run.smithi046.stdout:4807 trunc from 0x6fc0611 to 0xaea0830 2024-09-12T07:20:54.885 INFO:teuthology.orchestra.run.smithi046.stdout:4809 read 0x99449e8 thru 0x9948916 (0x3f2f bytes) 2024-09-12T07:20:54.885 INFO:teuthology.orchestra.run.smithi046.stdout:4811 punch from 0x7620a34 to 0x76308d9, (0xfea5 bytes) 2024-09-12T07:20:54.885 INFO:teuthology.orchestra.run.smithi046.stdout:4812 trunc from 0xaea0830 to 0x7e1cdbd 2024-09-12T07:20:54.885 INFO:teuthology.orchestra.run.smithi046.stdout:4813 punch from 0x38cd597 to 0x38d4b41, (0x75aa bytes) 2024-09-12T07:20:54.885 INFO:teuthology.orchestra.run.smithi046.stdout:4814 trunc from 0x7e1cdbd to 0x4cbf4a9 2024-09-12T07:20:54.885 INFO:teuthology.orchestra.run.smithi046.stdout:4815 read 0x147e995 thru 0x14836e2 (0x4d4e bytes) 2024-09-12T07:20:54.885 INFO:teuthology.orchestra.run.smithi046.stdout:4817 punch from 0xc0b33d to 0xc14187, (0x8e4a bytes) 2024-09-12T07:20:54.886 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:54.886 INFO:teuthology.orchestra.run.smithi046.stdout:4819 writesame 0x42f43c6 thru 0x42fc86b data_size 0x1(0x84a6 bytes) 2024-09-12T07:20:54.897 INFO:teuthology.orchestra.run.smithi046.stdout:4820 write 0x58583d3 thru 0x585c6e7 (0x4315 bytes) 2024-09-12T07:20:54.899 INFO:teuthology.orchestra.run.smithi046.stdout:4824 read 0x48fc594 thru 0x4902bfb (0x6668 bytes) 2024-09-12T07:20:54.899 INFO:teuthology.orchestra.run.smithi046.stdout:4825 writesame 0x1dbc6dc thru 0x1dc46b1 data_size 0x1(0x7fd6 bytes) 2024-09-12T07:20:54.910 INFO:teuthology.orchestra.run.smithi046.stdout:4827 read 0x228410f thru 0x228de34 (0x9d26 bytes) 2024-09-12T07:20:54.917 INFO:teuthology.orchestra.run.smithi046.stdout:4828 trunc from 0x585c6e8 to 0xbf2ab54 2024-09-12T07:20:54.923 INFO:teuthology.orchestra.run.smithi046.stdout:4829 read 0x1e2cff4 thru 0x1e3c9f9 (0xfa06 bytes) 2024-09-12T07:20:54.923 INFO:teuthology.orchestra.run.smithi046.stdout:4832 writesame 0x56580b2 thru 0x565d494 data_size 0x5(0x53e3 bytes) 2024-09-12T07:20:54.926 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:54.926 INFO:teuthology.orchestra.run.smithi046.stdout:4837 punch from 0x566cc1e to 0x567acca, (0xe0ac bytes) 2024-09-12T07:20:54.931 INFO:teuthology.orchestra.run.smithi046.stdout:4839 write 0xc8386f0 thru 0xc83f58f (0x6ea0 bytes) 2024-09-12T07:20:54.935 INFO:teuthology.orchestra.run.smithi046.stdout:4842 trunc from 0xc83f590 to 0xcdebf7b 2024-09-12T07:20:54.949 INFO:teuthology.orchestra.run.smithi046.stdout:4843 trunc from 0xcdebf7b to 0x4190e1e 2024-09-12T07:20:54.961 INFO:teuthology.orchestra.run.smithi046.stdout:4845 punch from 0x2e9fddd to 0x2ea7525, (0x7748 bytes) 2024-09-12T07:20:54.961 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:54.961 INFO:teuthology.orchestra.run.smithi046.stdout:4848 read 0x2d76616 thru 0x2d788f5 (0x22e0 bytes) 2024-09-12T07:20:54.961 INFO:teuthology.orchestra.run.smithi046.stdout:4849 writesame 0x1dcadd5 thru 0x1dd81ef data_size 0x1(0xd41b bytes) 2024-09-12T07:20:54.980 INFO:teuthology.orchestra.run.smithi046.stdout:4850 read 0x23c8ac6 thru 0x23ce7b0 (0x5ceb bytes) 2024-09-12T07:20:54.980 INFO:teuthology.orchestra.run.smithi046.stdout:4851 writesame 0xba1df9 thru 0xba34e0 data_size 0x4(0x16e8 bytes) 2024-09-12T07:20:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:4852 punch from 0x7eb342 to 0x7f397b, (0x8639 bytes) 2024-09-12T07:20:54.983 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:54.983+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:54.988 INFO:teuthology.orchestra.run.smithi046.stdout:4854 write 0x5030de1 thru 0x5039a7c (0x8c9c bytes) 2024-09-12T07:20:54.989 INFO:teuthology.orchestra.run.smithi046.stdout:4855 read 0x270b956 thru 0x2718aa9 (0xd154 bytes) 2024-09-12T07:20:54.998 INFO:teuthology.orchestra.run.smithi046.stdout:4857 write 0x69d7cfb thru 0x69de5d3 (0x68d9 bytes) 2024-09-12T07:20:55.003 INFO:teuthology.orchestra.run.smithi046.stdout:4858 read 0x3a6a565 thru 0x3a6f29c (0x4d38 bytes) 2024-09-12T07:20:55.013 INFO:teuthology.orchestra.run.smithi046.stdout:4859 write 0xa23d871 thru 0xa24a215 (0xc9a5 bytes) 2024-09-12T07:20:55.018 INFO:teuthology.orchestra.run.smithi046.stdout:4863 read 0x3785923 thru 0x378cee9 (0x75c7 bytes) 2024-09-12T07:20:55.018 INFO:teuthology.orchestra.run.smithi046.stdout:4864 write 0x168c287 thru 0x168ccdc (0xa56 bytes) 2024-09-12T07:20:55.022 INFO:teuthology.orchestra.run.smithi046.stdout:4865 trunc from 0xa24a216 to 0x8305d1e 2024-09-12T07:20:55.031 INFO:teuthology.orchestra.run.smithi046.stdout:4866 punch from 0x5bad233 to 0x5bb223f, (0x500c bytes) 2024-09-12T07:20:55.032 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:55.032 INFO:teuthology.orchestra.run.smithi046.stdout:4870 writesame 0x80d73eb thru 0x80e2b86 data_size 0x6(0xb79c bytes) 2024-09-12T07:20:55.041 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:55.041 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:55.041 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:55.041 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:55.049 INFO:teuthology.orchestra.run.smithi046.stdout:4871 write 0xb381a4b thru 0xb383cdf (0x2295 bytes) 2024-09-12T07:20:55.050 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:55.052 INFO:teuthology.orchestra.run.smithi046.stdout:4873 read 0x9b5be6e thru 0x9b69d2d (0xdec0 bytes) 2024-09-12T07:20:55.053 INFO:teuthology.orchestra.run.smithi046.stdout:4874 read 0xa4dacf9 thru 0xa4dda6f (0x2d77 bytes) 2024-09-12T07:20:55.058 INFO:teuthology.orchestra.run.smithi046.stdout:4877 write 0xb4c6ba7 thru 0xb4ccd56 (0x61b0 bytes) 2024-09-12T07:20:55.062 INFO:teuthology.orchestra.run.smithi046.stdout:4878 write 0x8c06ada thru 0x8c0a25e (0x3785 bytes) 2024-09-12T07:20:55.066 INFO:teuthology.orchestra.run.smithi046.stdout:4879 read 0x7c7b5b1 thru 0x7c8602a (0xaa7a bytes) 2024-09-12T07:20:55.071 INFO:teuthology.orchestra.run.smithi046.stdout:4880 write 0xb99fe65 thru 0xb9af9c7 (0xfb63 bytes) 2024-09-12T07:20:55.076 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:55.076 INFO:teuthology.orchestra.run.smithi046.stdout:4883 write 0x2c6c725 thru 0x2c7bcb2 (0xf58e bytes) 2024-09-12T07:20:55.085 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:55.093 INFO:teuthology.orchestra.run.smithi046.stdout:4888 write 0xd7de26d thru 0xd7deebc (0xc50 bytes) 2024-09-12T07:20:55.094 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:55.096+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:55.096 INFO:teuthology.orchestra.run.smithi046.stdout:4889 punch from 0xd490132 to 0xd49fe11, (0xfcdf bytes) 2024-09-12T07:20:55.096 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:55.096 INFO:teuthology.orchestra.run.smithi046.stdout:4892 write 0xdf4847 thru 0xdf81d2 (0x398c bytes) 2024-09-12T07:20:55.100 INFO:teuthology.orchestra.run.smithi046.stdout:4893 read 0x4040838 thru 0x40454cc (0x4c95 bytes) 2024-09-12T07:20:55.100 INFO:teuthology.orchestra.run.smithi046.stdout:4895 punch from 0x4685dd4 to 0x4690cf8, (0xaf24 bytes) 2024-09-12T07:20:55.100 INFO:teuthology.orchestra.run.smithi046.stdout:4896 write 0x3cbe0a9 thru 0x3cbeee9 (0xe41 bytes) 2024-09-12T07:20:55.102 INFO:teuthology.orchestra.run.smithi046.stdout:4897 punch from 0xd1b0033 to 0xd1b145e, (0x142b bytes) 2024-09-12T07:20:55.102 INFO:teuthology.orchestra.run.smithi046.stdout:4898 read 0x5c4a356 thru 0x5c5802b (0xdcd6 bytes) 2024-09-12T07:20:55.109 INFO:teuthology.orchestra.run.smithi046.stdout:4899 trunc from 0xd7deebd to 0x752c55c 2024-09-12T07:20:55.130 INFO:teuthology.orchestra.run.smithi046.stdout:4900 writesame 0xa2ce901 thru 0xa2d9970 data_size 0x18(0xb070 bytes) 2024-09-12T07:20:55.141 INFO:teuthology.orchestra.run.smithi046.stdout:4902 trunc from 0xa2d9971 to 0x14f2718 2024-09-12T07:20:55.166 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:55.166 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:55.166 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:55.166 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:55.166 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:55.173 INFO:teuthology.orchestra.run.smithi046.stdout:4903 writesame 0xecf50c8 thru 0xed03a08 data_size 0x1(0xe941 bytes) 2024-09-12T07:20:55.175 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:55.187 INFO:teuthology.orchestra.run.smithi046.stdout:4904 trunc from 0xed03a09 to 0x269a365 2024-09-12T07:20:55.213 INFO:teuthology.orchestra.run.smithi046.stdout:4906 writesame 0xe98e2e2 thru 0xe992848 data_size 0x1(0x4567 bytes) 2024-09-12T07:20:55.213 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:55.214 INFO:teuthology.orchestra.run.smithi046.stdout:4909 read 0x94f39d7 thru 0x94fd1d1 (0x97fb bytes) 2024-09-12T07:20:55.214 INFO:teuthology.orchestra.run.smithi046.stdout:4910 writesame 0xc55d5ee thru 0xc5606f6 data_size 0x1(0x3109 bytes) 2024-09-12T07:20:55.216 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:55.218+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:55.222 INFO:teuthology.orchestra.run.smithi046.stdout:4911 punch from 0x7969701 to 0x7974b6d, (0xb46c bytes) 2024-09-12T07:20:55.222 INFO:teuthology.orchestra.run.smithi046.stdout:4912 write 0x6f8d55f thru 0x6f8ffd5 (0x2a77 bytes) 2024-09-12T07:20:55.225 INFO:teuthology.orchestra.run.smithi046.stdout:4913 write 0xcfc7d0b thru 0xcfccf26 (0x521c bytes) 2024-09-12T07:20:55.231 INFO:teuthology.orchestra.run.smithi046.stdout:4914 clone 40 order 23 su 32768 sc 3 2024-09-12T07:20:55.285 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:55.286+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:55.324 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:55.324 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.843751894999997, 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:20:55.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:20:55.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:20:55.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:20:55.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.329 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.843725014999997, 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:20:55.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:20:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.843700994999999, 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.334 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:55.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.843673348999999, 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.339 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:20:55.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:20:55.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:20:55.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:20:55.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:20:55.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:20:55.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:20:55.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.843649886999998, 2024-09-12T07:20:55.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:55.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:20:55.347 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.843627359999999, 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.348 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:20:55.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.843605556999997, 2024-09-12T07:20:55.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:55.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:20:55.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:20:55.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:20:55.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:20:55.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:20:55.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:20:55.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.836971777000002, 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.357 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:20:55.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.836951769000002, 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:20:55.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:20:55.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.365 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.830468736999997, 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.366 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:20:55.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.830433450000001, 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:55.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.830410817000001, 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:20:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.830356762000001, 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:20:55.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.830337821000001, 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:20:55.384 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:20:55.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:20:55.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:20:55.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:20:55.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:20:55.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.830325883999997, 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.389 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:55.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:20:55.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.830314268999999, 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:20:55.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:20:55.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:20:55.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:20:55.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:20:55.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:20:55.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:20:55.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.823301661000002, 2024-09-12T07:20:55.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:55.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.402 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.823277945999997, 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:20:55.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.823147089000003, 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.859476299000001, 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.411 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:55.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:20:55.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:20:55.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:20:55.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:20:55.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.420 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.421 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:55.422 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.7 to 1 (10000) 2024-09-12T07:20:55.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:55.423 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:55.423+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:55.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T07:20:55.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-12T07:20:55.492 INFO:teuthology.orchestra.run.smithi046.stdout:leaving image image_client.0-clone39 intact 2024-09-12T07:20:55.523 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:55.525+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:55.572 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:55.572 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:55.572 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:55.572 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:55.580 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:55.623 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:55.625+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:55.705 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-12T07:20:55.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:20:55.724 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:55.726+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:55.729 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:55.729 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:55.729 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:55.730 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:55.730 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:55.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:55.828 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:55.825+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:55.867 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:55.884 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:55.885 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:55.889 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:55.889 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:55.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:55.889 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.360791970000001, 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:20:55.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:20:55.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:20:55.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.892 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.892 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.892 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:20:55.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:20:55.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.892 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:20:55.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.895 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.508994303999998, 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:20:55.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:20:55.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:20:55.900 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.900 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.900 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.900 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.900 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:55.900 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.508871026999998, 2024-09-12T07:20:55.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:20:55.900 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.900 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:20:55.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:20:55.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.904 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.508667635999998, 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:20:55.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:20:55.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.508531808999997, 2024-09-12T07:20:55.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:55.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:20:55.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:20:55.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.912 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.912 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:20:55.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:20:55.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.912 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.912 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:20:55.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:20:55.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:20:55.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.508205152999999, 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.914 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.916 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.917 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.918 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.508166265, 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:20:55.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:20:55.923 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:20:55.923 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.923 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.923 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.923 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.923 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:55.923 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.507799390999999, 2024-09-12T07:20:55.923 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:55.924 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.927 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.507664535000004, 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.928 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.929 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.507294643999998, 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:20:55.932 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.934 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.934 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:55.934 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.934 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.934 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:20:55.934 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:20:55.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:20:55.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.507104446, 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:55.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.506884366999998, 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:20:55.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:20:55.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.506740702000002, 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:55.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:20:55.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:20:55.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:20:55.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:20:55.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.950 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.506514529999997, 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.951 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:20:55.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:20:55.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.506314172000003, 2024-09-12T07:20:55.955 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:20:55.956 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.956 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.956 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.956 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.956 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.956 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.956 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:20:55.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.956 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:20:55.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:20:55.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.505906289000002, 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.960 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.961 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.961 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.961 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.961 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:20:55.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.961 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.961 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.961 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:55.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.963 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.505440407000002, 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.965 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:20:55.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:20:55.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:20:55.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:20:55.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:20:55.969 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.969 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.969 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.503838197, 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:20:55.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:20:55.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.503304161999999, 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.974 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:55.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.247106092000003, 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:55.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:55.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:55.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:55.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:55.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:20:55.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:20:55.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:55.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:55.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:55.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:20:55.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:20:55.983 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.983 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.983 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.983 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:55.983 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:55.983 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:55.984 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:55.985 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:55.985 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:55.985 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:55.986 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:55.986 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:55.986 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:55.987 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:55.987 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:55.987 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:55.988 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:55.988 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:55.986+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:55.988 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:55.988 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:56.013 INFO:teuthology.orchestra.run.smithi046.stdout:4915 read 0x9b16bf9 thru 0x9b1d019 (0x6421 bytes) 2024-09-12T07:20:56.030 INFO:teuthology.orchestra.run.smithi046.stdout:4916 trunc from 0xe992849 to 0xb143e1 2024-09-12T07:20:56.085 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:56.087+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:56.120 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:56.121 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:56.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:20:56.135 INFO:teuthology.orchestra.run.smithi046.stdout:4917 writesame 0xfad4ef thru 0xfb4325 data_size 0x21(0x6e37 bytes) 2024-09-12T07:20:56.137 INFO:teuthology.orchestra.run.smithi046.stdout:4918 read 0x149c6 thru 0x1527a (0x8b5 bytes) 2024-09-12T07:20:56.138 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.138 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.138 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.148 INFO:teuthology.orchestra.run.smithi046.stdout:4922 write 0x5baa543 thru 0x5baa802 (0x2c0 bytes) 2024-09-12T07:20:56.151 INFO:teuthology.orchestra.run.smithi046.stdout:4923 read 0x3d7b107 thru 0x3d81124 (0x601e bytes) 2024-09-12T07:20:56.164 INFO:teuthology.orchestra.run.smithi046.stdout:4925 write 0xbbf1f35 thru 0xbc00526 (0xe5f2 bytes) 2024-09-12T07:20:56.168 INFO:teuthology.orchestra.run.smithi046.stdout:4926 punch from 0x52e1a88 to 0x52ec14a, (0xa6c2 bytes) 2024-09-12T07:20:56.168 INFO:teuthology.orchestra.run.smithi046.stdout:4928 read 0xba38691 thru 0xba4008c (0x79fc bytes) 2024-09-12T07:20:56.176 INFO:teuthology.orchestra.run.smithi046.stdout:4930 trunc from 0xbc00527 to 0x50fc539 2024-09-12T07:20:56.186 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:56.188+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:56.188 INFO:teuthology.orchestra.run.smithi046.stdout:4931 trunc from 0x50fc539 to 0x6e686fc 2024-09-12T07:20:56.206 INFO:teuthology.orchestra.run.smithi046.stdout:4932 write 0xd1f25e1 thru 0xd200d29 (0xe749 bytes) 2024-09-12T07:20:56.211 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.211 INFO:teuthology.orchestra.run.smithi046.stdout:4937 punch from 0x5039633 to 0x50397b1, (0x17e bytes) 2024-09-12T07:20:56.211 INFO:teuthology.orchestra.run.smithi046.stdout:4938 trunc from 0xd200d2a to 0xd74c6d5 2024-09-12T07:20:56.217 INFO:teuthology.orchestra.run.smithi046.stdout:4939 read 0x9e2cb6f thru 0x9e33732 (0x6bc4 bytes) 2024-09-12T07:20:56.217 INFO:teuthology.orchestra.run.smithi046.stdout:4940 read 0x824ab5 thru 0x829ce0 (0x522c bytes) 2024-09-12T07:20:56.217 INFO:teuthology.orchestra.run.smithi046.stdout:4941 read 0x7d6dae8 thru 0x7d7d949 (0xfe62 bytes) 2024-09-12T07:20:56.229 INFO:teuthology.orchestra.run.smithi046.stdout:4943 trunc from 0xd74c6d5 to 0x32e5609 2024-09-12T07:20:56.246 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:56.246 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:56.254 INFO:teuthology.orchestra.run.smithi046.stdout:4944 writesame 0xb6b01bf thru 0xb6b496c data_size 0x19(0x47ae bytes) 2024-09-12T07:20:56.254 INFO:teuthology.orchestra.run.smithi046.stdout:4945 write 0x978ead9 thru 0x979482d (0x5d55 bytes) 2024-09-12T07:20:56.256 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:20:56.258 INFO:teuthology.orchestra.run.smithi046.stdout:4946 read 0x161150 thru 0x167acf (0x6980 bytes) 2024-09-12T07:20:56.258 INFO:teuthology.orchestra.run.smithi046.stdout:4948 read 0xf9c43a thru 0xfa85a6 (0xc16d bytes) 2024-09-12T07:20:56.259 INFO:teuthology.orchestra.run.smithi046.stdout:4949 punch from 0x8d20ac5 to 0x8d2514d, (0x4688 bytes) 2024-09-12T07:20:56.259 INFO:teuthology.orchestra.run.smithi046.stdout:4950 punch from 0x8ade338 to 0x8ae79a5, (0x966d bytes) 2024-09-12T07:20:56.259 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.259 INFO:teuthology.orchestra.run.smithi046.stdout:4952 punch from 0x33ab0c9 to 0x33b6d97, (0xbcce bytes) 2024-09-12T07:20:56.259 INFO:teuthology.orchestra.run.smithi046.stdout:4953 write 0x4aa0fed thru 0x4aa2092 (0x10a6 bytes) 2024-09-12T07:20:56.262 INFO:teuthology.orchestra.run.smithi046.stdout:4955 read 0x9e325d1 thru 0x9e38099 (0x5ac9 bytes) 2024-09-12T07:20:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:4957 writesame 0x5b3d19c thru 0x5b3f422 data_size 0x1(0x2287 bytes) 2024-09-12T07:20:56.266 INFO:teuthology.orchestra.run.smithi046.stdout:4958 write 0xaa1b9a8 thru 0xaa210fb (0x5754 bytes) 2024-09-12T07:20:56.270 INFO:teuthology.orchestra.run.smithi046.stdout:4959 punch from 0x6c220c2 to 0x6c2e65f, (0xc59d bytes) 2024-09-12T07:20:56.270 INFO:teuthology.orchestra.run.smithi046.stdout:4960 read 0x185cda7 thru 0x1869b65 (0xcdbf bytes) 2024-09-12T07:20:56.279 INFO:teuthology.orchestra.run.smithi046.stdout:4963 trunc from 0xb6b496d to 0x473a98b 2024-09-12T07:20:56.287 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:56.288+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:56.304 INFO:teuthology.orchestra.run.smithi046.stdout:4964 write 0xdda2c4f thru 0xdda3bfc (0xfae bytes) 2024-09-12T07:20:56.308 INFO:teuthology.orchestra.run.smithi046.stdout:4967 read 0x6e2488c thru 0x6e2ba42 (0x71b7 bytes) 2024-09-12T07:20:56.308 INFO:teuthology.orchestra.run.smithi046.stdout:4968 write 0x9476988 thru 0x9480960 (0x9fd9 bytes) 2024-09-12T07:20:56.318 INFO:teuthology.orchestra.run.smithi046.stdout:4969 trunc from 0xdda3bfd to 0x89092fd 2024-09-12T07:20:56.330 INFO:teuthology.orchestra.run.smithi046.stdout:4971 punch from 0x6f2a744 to 0x6f39c2b, (0xf4e7 bytes) 2024-09-12T07:20:56.330 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.330 INFO:teuthology.orchestra.run.smithi046.stdout:4973 write 0x6aeee64 thru 0x6af3e13 (0x4fb0 bytes) 2024-09-12T07:20:56.341 INFO:teuthology.orchestra.run.smithi046.stdout:4974 trunc from 0x89092fd to 0x251dc07 2024-09-12T07:20:56.353 INFO:teuthology.orchestra.run.smithi046.stdout:4975 trunc from 0x251dc07 to 0x4660601 2024-09-12T07:20:56.368 INFO:teuthology.orchestra.run.smithi046.stdout:4976 trunc from 0x4660601 to 0xd4dc71b 2024-09-12T07:20:56.374 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.374 INFO:teuthology.orchestra.run.smithi046.stdout:4978 write 0x95dfb70 thru 0x95e4233 (0x46c4 bytes) 2024-09-12T07:20:56.378 INFO:teuthology.orchestra.run.smithi046.stdout:4979 write 0x1f6e086 thru 0x1f77468 (0x93e3 bytes) 2024-09-12T07:20:56.388 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:56.390+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:56.388 INFO:teuthology.orchestra.run.smithi046.stdout:4980 writesame 0xec95898 thru 0xec9b97a data_size 0x1(0x60e3 bytes) 2024-09-12T07:20:56.404 INFO:teuthology.orchestra.run.smithi046.stdout:4981 trunc from 0xec9b97b to 0xc1a80b 2024-09-12T07:20:56.409 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:56.409 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:56.415 INFO:teuthology.orchestra.run.smithi046.stdout:4982 read 0x283629 thru 0x283969 (0x341 bytes) 2024-09-12T07:20:56.416 INFO:teuthology.orchestra.run.smithi046.stdout:4983 read 0xa331fb thru 0xa3def5 (0xacfb bytes) 2024-09-12T07:20:56.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:20:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.3 to 1 (10000) 2024-09-12T07:20:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is set 2024-09-12T07:20:56.437 INFO:teuthology.orchestra.run.smithi046.stdout:4984 writesame 0x96eda04 thru 0x96efddf data_size 0x24(0x23dc bytes) 2024-09-12T07:20:56.437 INFO:teuthology.orchestra.run.smithi046.stdout:4987 punch from 0x51da70c to 0x51da87d, (0x171 bytes) 2024-09-12T07:20:56.437 INFO:teuthology.orchestra.run.smithi046.stdout:4988 trunc from 0x96efde0 to 0x9e18886 2024-09-12T07:20:56.441 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-12T07:20:56.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:20:56.444 INFO:teuthology.orchestra.run.smithi046.stdout:4989 write 0x67bd83b thru 0x67cad3e (0xd504 bytes) 2024-09-12T07:20:56.450 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.450 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.453 INFO:teuthology.orchestra.run.smithi046.stdout:4995 trunc from 0x9e18886 to 0xc3b5e28 2024-09-12T07:20:56.458 INFO:teuthology.orchestra.run.smithi046.stdout:4996 read 0x943768f thru 0x943f3a1 (0x7d13 bytes) 2024-09-12T07:20:56.458 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.459 INFO:teuthology.orchestra.run.smithi046.stdout:4998 read 0x53d771b thru 0x53df726 (0x800c bytes) 2024-09-12T07:20:56.459 INFO:teuthology.orchestra.run.smithi046.stdout:4999 read 0x1a790ad thru 0x1a85d14 (0xcc68 bytes) 2024-09-12T07:20:56.459 INFO:teuthology.orchestra.run.smithi046.stdout:5000 write 0x9a05a16 thru 0x9a07e22 (0x240d bytes) 2024-09-12T07:20:56.461 INFO:teuthology.orchestra.run.smithi046.stdout:5001 read 0x4b8b9e5 thru 0x4b91872 (0x5e8e bytes) 2024-09-12T07:20:56.461 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.468 INFO:teuthology.orchestra.run.smithi046.stdout:5003 write 0xdd1c0c6 thru 0xdd202fd (0x4238 bytes) 2024-09-12T07:20:56.473 INFO:teuthology.orchestra.run.smithi046.stdout:5004 read 0x4df7313 thru 0x4df8bcd (0x18bb bytes) 2024-09-12T07:20:56.473 INFO:teuthology.orchestra.run.smithi046.stdout:5005 write 0xa33f8cc thru 0xa342e78 (0x35ad bytes) 2024-09-12T07:20:56.477 INFO:teuthology.orchestra.run.smithi046.stdout:5007 read 0xb70ce0d thru 0xb711f6e (0x5162 bytes) 2024-09-12T07:20:56.477 INFO:teuthology.orchestra.run.smithi046.stdout:5008 punch from 0x19bb71 to 0x1a39bb, (0x7e4a bytes) 2024-09-12T07:20:56.477 INFO:teuthology.orchestra.run.smithi046.stdout:5009 read 0x11e5052 thru 0x11ef4d2 (0xa481 bytes) 2024-09-12T07:20:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:5010 read 0x791dfe5 thru 0x792d2c4 (0xf2e0 bytes) 2024-09-12T07:20:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:5012 read 0x3574778 thru 0x357b7dd (0x7066 bytes) 2024-09-12T07:20:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:5015 write 0x9f8485e thru 0x9f8874e (0x3ef1 bytes) 2024-09-12T07:20:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:5016 punch from 0x9ac67a1 to 0x9ad33dc, (0xcc3b bytes) 2024-09-12T07:20:56.483 INFO:teuthology.orchestra.run.smithi046.stdout:5017 trunc from 0xdd202fe to 0xb2fb4eb 2024-09-12T07:20:56.489 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:56.491+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:56.505 INFO:teuthology.orchestra.run.smithi046.stdout:5018 trunc from 0xb2fb4eb to 0x767bc6 2024-09-12T07:20:56.521 INFO:teuthology.orchestra.run.smithi046.stdout:5019 punch from 0x953de to 0x99412, (0x4034 bytes) 2024-09-12T07:20:56.539 INFO:teuthology.orchestra.run.smithi046.stdout:5020 write 0xb0d6182 thru 0xb0e10f6 (0xaf75 bytes) 2024-09-12T07:20:56.544 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.544 INFO:teuthology.orchestra.run.smithi046.stdout:5022 read 0x708292e thru 0x7085abb (0x318e bytes) 2024-09-12T07:20:56.544 INFO:teuthology.orchestra.run.smithi046.stdout:5024 read 0x9686c95 thru 0x968c8b5 (0x5c21 bytes) 2024-09-12T07:20:56.544 INFO:teuthology.orchestra.run.smithi046.stdout:5026 writesame 0x79a2b17 thru 0x79ad174 data_size 0x5(0xa65e bytes) 2024-09-12T07:20:56.550 INFO:teuthology.orchestra.run.smithi046.stdout:5027 writesame 0x947a863 thru 0x9482225 data_size 0x7(0x79c3 bytes) 2024-09-12T07:20:56.555 INFO:teuthology.orchestra.run.smithi046.stdout:5030 write 0xaa6822c thru 0xaa7512f (0xcf04 bytes) 2024-09-12T07:20:56.557 INFO:teuthology.orchestra.run.smithi046.stdout:5032 writesame 0x8a672f1 thru 0x8a7505b data_size 0x1(0xdd6b bytes) 2024-09-12T07:20:56.558 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:56.558 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:56.570 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:20:56.582 INFO:teuthology.orchestra.run.smithi046.stdout:5033 read 0x5204f8d thru 0x520598a (0x9fe bytes) 2024-09-12T07:20:56.582 INFO:teuthology.orchestra.run.smithi046.stdout:5035 writesame 0x770eed7 thru 0x7716b3c data_size 0x1(0x7c66 bytes) 2024-09-12T07:20:56.590 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:56.592+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:56.609 INFO:teuthology.orchestra.run.smithi046.stdout:5036 trunc from 0xb0e10f7 to 0x2ce3504 2024-09-12T07:20:56.620 INFO:teuthology.orchestra.run.smithi046.stdout:5038 punch from 0xb1d388 to 0xb208c4, (0x353c bytes) 2024-09-12T07:20:56.620 INFO:teuthology.orchestra.run.smithi046.stdout:5039 read 0x2440336 thru 0x2444d96 (0x4a61 bytes) 2024-09-12T07:20:56.620 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.640 INFO:teuthology.orchestra.run.smithi046.stdout:5044 write 0xd77212f thru 0xd7755dd (0x34af bytes) 2024-09-12T07:20:56.643 INFO:teuthology.orchestra.run.smithi046.stdout:5045 writesame 0xc9d53dd thru 0xc9d77d0 data_size 0xd(0x23f4 bytes) 2024-09-12T07:20:56.646 INFO:teuthology.orchestra.run.smithi046.stdout:5046 write 0x4bc280e thru 0x4bd1723 (0xef16 bytes) 2024-09-12T07:20:56.654 INFO:teuthology.orchestra.run.smithi046.stdout:5049 read 0x5338f24 thru 0x533ecd5 (0x5db2 bytes) 2024-09-12T07:20:56.654 INFO:teuthology.orchestra.run.smithi046.stdout:5051 read 0x8477e67 thru 0x8479a36 (0x1bd0 bytes) 2024-09-12T07:20:56.654 INFO:teuthology.orchestra.run.smithi046.stdout:5053 write 0x4f56db0 thru 0x4f57845 (0xa96 bytes) 2024-09-12T07:20:56.654 INFO:teuthology.orchestra.run.smithi046.stdout:5054 write 0x530e846 thru 0x531b329 (0xcae4 bytes) 2024-09-12T07:20:56.654 INFO:teuthology.orchestra.run.smithi046.stdout:5055 writesame 0xc4a32b thru 0xc50bf1 data_size 0x1(0x68c7 bytes) 2024-09-12T07:20:56.665 INFO:teuthology.orchestra.run.smithi046.stdout:5056 read 0x9c71afc thru 0x9c7910b (0x7610 bytes) 2024-09-12T07:20:56.670 INFO:teuthology.orchestra.run.smithi046.stdout:5057 trunc from 0xd7755de to 0xa5201c4 2024-09-12T07:20:56.685 INFO:teuthology.orchestra.run.smithi046.stdout:5058 trunc from 0xa5201c4 to 0x6ac3420 2024-09-12T07:20:56.690 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:56.692+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:56.692 INFO:teuthology.orchestra.run.smithi046.stdout:5059 read 0x251631d thru 0x25205dd (0xa2c1 bytes) 2024-09-12T07:20:56.705 INFO:teuthology.orchestra.run.smithi046.stdout:5060 writesame 0xbcf344e thru 0xbcfacb8 data_size 0x1(0x786b bytes) 2024-09-12T07:20:56.705 INFO:teuthology.orchestra.run.smithi046.stdout:5061 write 0x7c1ee70 thru 0x7c26f5a (0x80eb bytes) 2024-09-12T07:20:56.709 INFO:teuthology.orchestra.run.smithi046.stdout:5062 read 0x6435005 thru 0x6440a83 (0xba7f bytes) 2024-09-12T07:20:56.710 INFO:teuthology.orchestra.run.smithi046.stdout:5063 writesame 0x92f3093 thru 0x9302d8b data_size 0x1(0xfcf9 bytes) 2024-09-12T07:20:56.713 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:56.713 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:56.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:20:56.731 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.731 INFO:teuthology.orchestra.run.smithi046.stdout:5065 punch from 0x4753dc8 to 0x475d2db, (0x9513 bytes) 2024-09-12T07:20:56.731 INFO:teuthology.orchestra.run.smithi046.stdout:5066 read 0x4a5c22a thru 0x4a61355 (0x512c bytes) 2024-09-12T07:20:56.732 INFO:teuthology.orchestra.run.smithi046.stdout:5068 punch from 0xf29227 to 0xf331f9, (0x9fd2 bytes) 2024-09-12T07:20:56.732 INFO:teuthology.orchestra.run.smithi046.stdout:5069 read 0x5640a0a thru 0x564c50f (0xbb06 bytes) 2024-09-12T07:20:56.732 INFO:teuthology.orchestra.run.smithi046.stdout:5071 read 0x6b1b600 thru 0x6b20056 (0x4a57 bytes) 2024-09-12T07:20:56.732 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.733 INFO:teuthology.orchestra.run.smithi046.stdout:5074 write 0x89ac797 thru 0x89bc415 (0xfc7f bytes) 2024-09-12T07:20:56.735 INFO:teuthology.orchestra.run.smithi046.stdout:5075 writesame 0x60aa666 thru 0x60addc8 data_size 0x1(0x3763 bytes) 2024-09-12T07:20:56.745 INFO:teuthology.orchestra.run.smithi046.stdout:5076 trunc from 0xbcfacb9 to 0xc544316 2024-09-12T07:20:56.750 INFO:teuthology.orchestra.run.smithi046.stdout:5077 read 0x442b7e3 thru 0x442d44e (0x1c6c bytes) 2024-09-12T07:20:56.750 INFO:teuthology.orchestra.run.smithi046.stdout:5078 read 0x9fa6316 thru 0x9fafedb (0x9bc6 bytes) 2024-09-12T07:20:56.751 INFO:teuthology.orchestra.run.smithi046.stdout:5079 read 0x1a7f642 thru 0x1a890a6 (0x9a65 bytes) 2024-09-12T07:20:56.751 INFO:teuthology.orchestra.run.smithi046.stdout:5081 read 0x3631e05 thru 0x363a2c7 (0x84c3 bytes) 2024-09-12T07:20:56.751 INFO:teuthology.orchestra.run.smithi046.stdout:5082 punch from 0x2abd22d to 0x2abef0e, (0x1ce1 bytes) 2024-09-12T07:20:56.751 INFO:teuthology.orchestra.run.smithi046.stdout:5083 read 0x64b1202 thru 0x64ba3db (0x91da bytes) 2024-09-12T07:20:56.757 INFO:teuthology.orchestra.run.smithi046.stdout:5084 trunc from 0xc544316 to 0x7f9c3f7 2024-09-12T07:20:56.774 INFO:teuthology.orchestra.run.smithi046.stdout:5085 trunc from 0x7f9c3f7 to 0x3e2fea7 2024-09-12T07:20:56.792 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:56.793+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:56.793 INFO:teuthology.orchestra.run.smithi046.stdout:5088 trunc from 0x3e2fea7 to 0x99c80ad 2024-09-12T07:20:56.810 INFO:teuthology.orchestra.run.smithi046.stdout:5089 trunc from 0x99c80ad to 0x736d5f 2024-09-12T07:20:56.817 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.817 INFO:teuthology.orchestra.run.smithi046.stdout:5092 punch from 0x2b187a to 0x2b5800, (0x3f86 bytes) 2024-09-12T07:20:56.817 INFO:teuthology.orchestra.run.smithi046.stdout:5093 read 0x263193 thru 0x2676a1 (0x450f bytes) 2024-09-12T07:20:56.826 INFO:teuthology.orchestra.run.smithi046.stdout:5094 write 0x280a607 thru 0x280d9a0 (0x339a bytes) 2024-09-12T07:20:56.828 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:20:56.828 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:20:56.830 INFO:teuthology.orchestra.run.smithi046.stdout:5096 read 0x16675d2 thru 0x16701a6 (0x8bd5 bytes) 2024-09-12T07:20:56.830 INFO:teuthology.orchestra.run.smithi046.stdout:5098 punch from 0x26a57d4 to 0x26acdbc, (0x75e8 bytes) 2024-09-12T07:20:56.830 INFO:teuthology.orchestra.run.smithi046.stdout:5099 write 0x1a5b58e thru 0x1a653cb (0x9e3e bytes) 2024-09-12T07:20:56.833 INFO:teuthology.orchestra.run.smithi046.stdout:5100 read 0x219a6a7 thru 0x21a0987 (0x62e1 bytes) 2024-09-12T07:20:56.833 INFO:teuthology.orchestra.run.smithi046.stdout:5101 read 0x146db6c thru 0x14785ff (0xaa94 bytes) 2024-09-12T07:20:56.834 INFO:teuthology.orchestra.run.smithi046.stdout:5102 trunc from 0x280d9a1 to 0x1b7c3df 2024-09-12T07:20:56.848 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-12T07:20:56.420130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1290361941,"num_objects":1518,"num_object_clones":367,"num_object_copies":3036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1518,"num_whiteouts":630,"num_read":190454,"num_read_kb":926088,"num_write":14001,"num_write_kb":494956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":64106948,"num_keys_recovered":75,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11314,"ondisk_log_size":11314,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":922960,"kb_used_data":647076,"kb_used_omap":12,"kb_used_meta":275699,"kb_avail":754051760,"statfs":{"total":773094113280,"available":772149002240,"internally_reserved":0,"allocated":662605824,"data_stored":797646675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":282316337},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":88302728,"num_objects":57,"num_object_clones":4,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":5419,"num_read_kb":12237,"num_write":852,"num_write_kb":9065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023785"},"pg_stats":[{"pgid":"3.1f","version":"91'446","reported_seq":2323,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.486579+0000","last_change":"2024-09-12T07:19:44.455827+0000","last_active":"2024-09-12T07:20:52.486579+0000","last_peered":"2024-09-12T07:20:52.486579+0000","last_clean":"2024-09-12T07:20:52.486579+0000","last_became_active":"2024-09-12T07:19:42.451212+0000","last_became_peered":"2024-09-12T07:19:42.451212+0000","last_unstale":"2024-09-12T07:20:52.486579+0000","last_undegraded":"2024-09-12T07:20:52.486579+0000","last_fullsized":"2024-09-12T07:20:52.486579+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:03:26.073069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001023672,"stat_sum":{"num_bytes":13840385,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":3701,"num_read_kb":16225,"num_write":620,"num_write_kb":6362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"90'292","reported_seq":1501,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.463172+0000","last_change":"2024-09-12T07:20:07.305432+0000","last_active":"2024-09-12T07:20:52.463172+0000","last_peered":"2024-09-12T07:20:52.463172+0000","last_clean":"2024-09-12T07:20:52.463172+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:20:52.463172+0000","last_undegraded":"2024-09-12T07:20:52.463172+0000","last_fullsized":"2024-09-12T07:20:52.463172+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:23.432336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081935099999999996,"stat_sum":{"num_bytes":58732364,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":18969,"num_read_kb":35160,"num_write":270,"num_write_kb":18916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"91'333","reported_seq":18517,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.553376+0000","last_change":"2024-09-12T07:19:44.464376+0000","last_active":"2024-09-12T07:20:52.553376+0000","last_peered":"2024-09-12T07:20:52.553376+0000","last_clean":"2024-09-12T07:20:52.553376+0000","last_became_active":"2024-09-12T07:19:42.451973+0000","last_became_peered":"2024-09-12T07:19:42.451973+0000","last_unstale":"2024-09-12T07:20:52.553376+0000","last_undegraded":"2024-09-12T07:20:52.553376+0000","last_fullsized":"2024-09-12T07:20:52.553376+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:14.774992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036949999999999,"stat_sum":{"num_bytes":41834087,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":22,"num_read":18658,"num_read_kb":31056,"num_write":384,"num_write_kb":12762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"90'440","reported_seq":2792,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397601+0000","last_change":"2024-09-12T07:19:44.464287+0000","last_active":"2024-09-12T07:20:52.397601+0000","last_peered":"2024-09-12T07:20:52.397601+0000","last_clean":"2024-09-12T07:20:52.397601+0000","last_became_active":"2024-09-12T07:19:42.455338+0000","last_became_peered":"2024-09-12T07:19:42.455338+0000","last_unstale":"2024-09-12T07:20:52.397601+0000","last_undegraded":"2024-09-12T07:20:52.397601+0000","last_fullsized":"2024-09-12T07:20:52.397601+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:21.916787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012347180000000001,"stat_sum":{"num_bytes":43084583,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":3357,"num_read_kb":15811,"num_write":600,"num_write_kb":8810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"91'305","reported_seq":17822,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.437869+0000","last_change":"2024-09-12T07:19:44.456652+0000","last_active":"2024-09-12T07:20:52.437869+0000","last_peered":"2024-09-12T07:20:52.437869+0000","last_clean":"2024-09-12T07:20:52.437869+0000","last_became_active":"2024-09-12T07:19:42.454990+0000","last_became_peered":"2024-09-12T07:19:42.454990+0000","last_unstale":"2024-09-12T07:20:52.437869+0000","last_undegraded":"2024-09-12T07:20:52.437869+0000","last_fullsized":"2024-09-12T07:20:52.437869+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:54.109441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049866,"stat_sum":{"num_bytes":44552147,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":18296,"num_read_kb":47346,"num_write":318,"num_write_kb":15496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"91'296","reported_seq":3196,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.544527+0000","last_change":"2024-09-12T07:19:44.459417+0000","last_active":"2024-09-12T07:20:52.544527+0000","last_peered":"2024-09-12T07:20:52.544527+0000","last_clean":"2024-09-12T07:20:52.544527+0000","last_became_active":"2024-09-12T07:19:42.454620+0000","last_became_peered":"2024-09-12T07:19:42.454620+0000","last_unstale":"2024-09-12T07:20:52.544527+0000","last_undegraded":"2024-09-12T07:20:52.544527+0000","last_fullsized":"2024-09-12T07:20:52.544527+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:36.284865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090501500000000001,"stat_sum":{"num_bytes":39289879,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":23,"num_read":3728,"num_read_kb":12271,"num_write":277,"num_write_kb":17032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"91'323","reported_seq":5006,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397461+0000","last_change":"2024-09-12T07:20:10.196744+0000","last_active":"2024-09-12T07:20:52.397461+0000","last_peered":"2024-09-12T07:20:52.397461+0000","last_clean":"2024-09-12T07:20:52.397461+0000","last_became_active":"2024-09-12T07:20:10.195312+0000","last_became_peered":"2024-09-12T07:20:10.195312+0000","last_unstale":"2024-09-12T07:20:52.397461+0000","last_undegraded":"2024-09-12T07:20:52.397461+0000","last_fullsized":"2024-09-12T07:20:52.397461+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:30:28.696660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113156,"stat_sum":{"num_bytes":34739920,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":15,"num_read":5309,"num_read_kb":38951,"num_write":425,"num_write_kb":12939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1101525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"91'473","reported_seq":2867,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.503709+0000","last_change":"2024-09-12T07:20:07.446518+0000","last_active":"2024-09-12T07:20:52.503709+0000","last_peered":"2024-09-12T07:20:52.503709+0000","last_clean":"2024-09-12T07:20:52.503709+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:20:52.503709+0000","last_undegraded":"2024-09-12T07:20:52.503709+0000","last_fullsized":"2024-09-12T07:20:52.503709+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:37.690882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000443294,"stat_sum":{"num_bytes":54337354,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":13,"num_read":5513,"num_read_kb":41321,"num_write":642,"num_write_kb":18029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"91'338","reported_seq":4975,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.532129+0000","last_change":"2024-09-12T07:19:44.456115+0000","last_active":"2024-09-12T07:20:52.532129+0000","last_peered":"2024-09-12T07:20:52.532129+0000","last_clean":"2024-09-12T07:20:52.532129+0000","last_became_active":"2024-09-12T07:19:42.452617+0000","last_became_peered":"2024-09-12T07:19:42.452617+0000","last_unstale":"2024-09-12T07:20:52.532129+0000","last_undegraded":"2024-09-12T07:20:52.532129+0000","last_fullsized":"2024-09-12T07:20:52.532129+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:29:53.445014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077251,"stat_sum":{"num_bytes":41864668,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":16,"num_read":5916,"num_read_kb":17020,"num_write":340,"num_write_kb":22272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"90'158","reported_seq":878,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397648+0000","last_change":"2024-09-12T07:19:44.463507+0000","last_active":"2024-09-12T07:20:52.397648+0000","last_peered":"2024-09-12T07:20:52.397648+0000","last_clean":"2024-09-12T07:20:52.397648+0000","last_became_active":"2024-09-12T07:19:42.455210+0000","last_became_peered":"2024-09-12T07:19:42.455210+0000","last_unstale":"2024-09-12T07:20:52.397648+0000","last_undegraded":"2024-09-12T07:20:52.397648+0000","last_fullsized":"2024-09-12T07:20:52.397648+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:10.840067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049753700000000004,"stat_sum":{"num_bytes":18453957,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":23,"num_read":616,"num_read_kb":7901,"num_write":161,"num_write_kb":6863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'192","reported_seq":6212,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505400+0000","last_change":"2024-09-12T07:20:07.261139+0000","last_active":"2024-09-12T07:20:52.505400+0000","last_peered":"2024-09-12T07:20:52.505400+0000","last_clean":"2024-09-12T07:20:52.505400+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:20:52.505400+0000","last_undegraded":"2024-09-12T07:20:52.505400+0000","last_fullsized":"2024-09-12T07:20:52.505400+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.425867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012225490000000001,"stat_sum":{"num_bytes":22445643,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":6416,"num_read_kb":43219,"num_write":167,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"89'285","reported_seq":4915,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505355+0000","last_change":"2024-09-12T07:19:44.481796+0000","last_active":"2024-09-12T07:20:52.505355+0000","last_peered":"2024-09-12T07:20:52.505355+0000","last_clean":"2024-09-12T07:20:52.505355+0000","last_became_active":"2024-09-12T07:19:42.452277+0000","last_became_peered":"2024-09-12T07:19:42.452277+0000","last_unstale":"2024-09-12T07:20:52.505355+0000","last_undegraded":"2024-09-12T07:20:52.505355+0000","last_fullsized":"2024-09-12T07:20:52.505355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:01.375693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012485160000000001,"stat_sum":{"num_bytes":57888336,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":19,"num_read":5159,"num_read_kb":60603,"num_write":313,"num_write_kb":30134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505578+0000","last_change":"2024-09-12T07:20:37.329418+0000","last_active":"2024-09-12T07:20:52.505578+0000","last_peered":"2024-09-12T07:20:52.505578+0000","last_clean":"2024-09-12T07:20:52.505578+0000","last_became_active":"2024-09-12T07:20:37.329145+0000","last_became_peered":"2024-09-12T07:20:37.329145+0000","last_unstale":"2024-09-12T07:20:52.505578+0000","last_undegraded":"2024-09-12T07:20:52.505578+0000","last_fullsized":"2024-09-12T07:20:52.505578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:58.555569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"89'429","reported_seq":1717,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.395525+0000","last_change":"2024-09-12T07:19:44.455257+0000","last_active":"2024-09-12T07:20:52.395525+0000","last_peered":"2024-09-12T07:20:52.395525+0000","last_clean":"2024-09-12T07:20:52.395525+0000","last_became_active":"2024-09-12T07:19:42.449730+0000","last_became_peered":"2024-09-12T07:19:42.449730+0000","last_unstale":"2024-09-12T07:20:52.395525+0000","last_undegraded":"2024-09-12T07:20:52.395525+0000","last_fullsized":"2024-09-12T07:20:52.395525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:16.258754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059259600000000003,"stat_sum":{"num_bytes":20649904,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":22,"num_read":2641,"num_read_kb":10111,"num_write":561,"num_write_kb":8756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397643+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:20:52.397643+0000","last_peered":"2024-09-12T07:20:52.397643+0000","last_clean":"2024-09-12T07:20:52.397643+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:20:52.397643+0000","last_undegraded":"2024-09-12T07:20:52.397643+0000","last_fullsized":"2024-09-12T07:20:52.397643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"91'360","reported_seq":1420,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.558392+0000","last_change":"2024-09-12T07:20:07.211023+0000","last_active":"2024-09-12T07:20:52.558392+0000","last_peered":"2024-09-12T07:20:52.558392+0000","last_clean":"2024-09-12T07:20:52.558392+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:20:52.558392+0000","last_undegraded":"2024-09-12T07:20:52.558392+0000","last_fullsized":"2024-09-12T07:20:52.558392+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:05.723251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087343100000000003,"stat_sum":{"num_bytes":44035048,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":1810,"num_read_kb":13214,"num_write":443,"num_write_kb":7845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505517+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:20:52.505517+0000","last_peered":"2024-09-12T07:20:52.505517+0000","last_clean":"2024-09-12T07:20:52.505517+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:20:52.505517+0000","last_undegraded":"2024-09-12T07:20:52.505517+0000","last_fullsized":"2024-09-12T07:20:52.505517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"88'340","reported_seq":1872,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397480+0000","last_change":"2024-09-12T07:19:44.458925+0000","last_active":"2024-09-12T07:20:52.397480+0000","last_peered":"2024-09-12T07:20:52.397480+0000","last_clean":"2024-09-12T07:20:52.397480+0000","last_became_active":"2024-09-12T07:19:42.452704+0000","last_became_peered":"2024-09-12T07:19:42.452704+0000","last_unstale":"2024-09-12T07:20:52.397480+0000","last_undegraded":"2024-09-12T07:20:52.397480+0000","last_fullsized":"2024-09-12T07:20:52.397480+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:17:15.336931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045577200000000001,"stat_sum":{"num_bytes":31139649,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":3043,"num_read_kb":10221,"num_write":441,"num_write_kb":6478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397540+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:20:52.397540+0000","last_peered":"2024-09-12T07:20:52.397540+0000","last_clean":"2024-09-12T07:20:52.397540+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:20:52.397540+0000","last_undegraded":"2024-09-12T07:20:52.397540+0000","last_fullsized":"2024-09-12T07:20:52.397540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"90'270","reported_seq":4678,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397767+0000","last_change":"2024-09-12T07:20:07.289858+0000","last_active":"2024-09-12T07:20:52.397767+0000","last_peered":"2024-09-12T07:20:52.397767+0000","last_clean":"2024-09-12T07:20:52.397767+0000","last_became_active":"2024-09-12T07:20:07.183819+0000","last_became_peered":"2024-09-12T07:20:07.183819+0000","last_unstale":"2024-09-12T07:20:52.397767+0000","last_undegraded":"2024-09-12T07:20:52.397767+0000","last_fullsized":"2024-09-12T07:20:52.397767+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:41:55.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015134619999999999,"stat_sum":{"num_bytes":59078065,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":23,"num_read":5060,"num_read_kb":50291,"num_write":347,"num_write_kb":20933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":3070354,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"91'200","reported_seq":2066,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.484934+0000","last_change":"2024-09-12T07:19:44.456465+0000","last_active":"2024-09-12T07:20:52.484934+0000","last_peered":"2024-09-12T07:20:52.484934+0000","last_clean":"2024-09-12T07:20:52.484934+0000","last_became_active":"2024-09-12T07:19:42.454103+0000","last_became_peered":"2024-09-12T07:19:42.454103+0000","last_unstale":"2024-09-12T07:20:52.484934+0000","last_undegraded":"2024-09-12T07:20:52.484934+0000","last_fullsized":"2024-09-12T07:20:52.484934+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:01:25.840939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010980880000000001,"stat_sum":{"num_bytes":23845044,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":26,"num_read":1747,"num_read_kb":7307,"num_write":182,"num_write_kb":8837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":160,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397578+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:20:52.397578+0000","last_peered":"2024-09-12T07:20:52.397578+0000","last_clean":"2024-09-12T07:20:52.397578+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:20:52.397578+0000","last_undegraded":"2024-09-12T07:20:52.397578+0000","last_fullsized":"2024-09-12T07:20:52.397578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"87'545","reported_seq":2475,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397355+0000","last_change":"2024-09-12T07:19:44.458868+0000","last_active":"2024-09-12T07:20:52.397355+0000","last_peered":"2024-09-12T07:20:52.397355+0000","last_clean":"2024-09-12T07:20:52.397355+0000","last_became_active":"2024-09-12T07:19:42.451807+0000","last_became_peered":"2024-09-12T07:19:42.451807+0000","last_unstale":"2024-09-12T07:20:52.397355+0000","last_undegraded":"2024-09-12T07:20:52.397355+0000","last_fullsized":"2024-09-12T07:20:52.397355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:58:02.768787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048363399999999998,"stat_sum":{"num_bytes":30205412,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":26,"num_read":4592,"num_read_kb":10908,"num_write":795,"num_write_kb":10412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T07:20:51.390827+0000","last_change":"2024-09-12T07:20:07.183543+0000","last_active":"2024-09-12T07:20:51.390827+0000","last_peered":"2024-09-12T07:20:51.390827+0000","last_clean":"2024-09-12T07:20:51.390827+0000","last_became_active":"2024-09-12T07:20:07.183213+0000","last_became_peered":"2024-09-12T07:20:07.183213+0000","last_unstale":"2024-09-12T07:20:51.390827+0000","last_undegraded":"2024-09-12T07:20:51.390827+0000","last_fullsized":"2024-09-12T07:20:51.390827+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.634459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"86'225","reported_seq":3380,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.395402+0000","last_change":"2024-09-12T07:19:44.455560+0000","last_active":"2024-09-12T07:20:52.395402+0000","last_peered":"2024-09-12T07:20:52.395402+0000","last_clean":"2024-09-12T07:20:52.395402+0000","last_became_active":"2024-09-12T07:19:42.451970+0000","last_became_peered":"2024-09-12T07:19:42.451970+0000","last_unstale":"2024-09-12T07:20:52.395402+0000","last_undegraded":"2024-09-12T07:20:52.395402+0000","last_fullsized":"2024-09-12T07:20:52.395402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:52.854392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083352100000000004,"stat_sum":{"num_bytes":45285524,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":4593,"num_read_kb":35426,"num_write":234,"num_write_kb":23622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505298+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:20:52.505298+0000","last_peered":"2024-09-12T07:20:52.505298+0000","last_clean":"2024-09-12T07:20:52.505298+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:20:52.505298+0000","last_undegraded":"2024-09-12T07:20:52.505298+0000","last_fullsized":"2024-09-12T07:20:52.505298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'874","reported_seq":3498,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.463057+0000","last_change":"2024-09-12T07:20:07.467030+0000","last_active":"2024-09-12T07:20:52.463057+0000","last_peered":"2024-09-12T07:20:52.463057+0000","last_clean":"2024-09-12T07:20:52.463057+0000","last_became_active":"2024-09-12T07:20:07.180577+0000","last_became_peered":"2024-09-12T07:20:07.180577+0000","last_unstale":"2024-09-12T07:20:52.463057+0000","last_undegraded":"2024-09-12T07:20:52.463057+0000","last_fullsized":"2024-09-12T07:20:52.463057+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:31:46.435494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089638599999999999,"stat_sum":{"num_bytes":49414846,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":25,"num_read":6286,"num_read_kb":40579,"num_write":1401,"num_write_kb":17418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":261251,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":159,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397589+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:20:52.397589+0000","last_peered":"2024-09-12T07:20:52.397589+0000","last_clean":"2024-09-12T07:20:52.397589+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:20:52.397589+0000","last_undegraded":"2024-09-12T07:20:52.397589+0000","last_fullsized":"2024-09-12T07:20:52.397589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":249,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505312+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:20:52.505312+0000","last_peered":"2024-09-12T07:20:52.505312+0000","last_clean":"2024-09-12T07:20:52.505312+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:20:52.505312+0000","last_undegraded":"2024-09-12T07:20:52.505312+0000","last_fullsized":"2024-09-12T07:20:52.505312+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"90'815","reported_seq":2017,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.463032+0000","last_change":"2024-09-12T07:20:07.428482+0000","last_active":"2024-09-12T07:20:52.463032+0000","last_peered":"2024-09-12T07:20:52.463032+0000","last_clean":"2024-09-12T07:20:52.463032+0000","last_became_active":"2024-09-12T07:20:07.180763+0000","last_became_peered":"2024-09-12T07:20:07.180763+0000","last_unstale":"2024-09-12T07:20:52.463032+0000","last_undegraded":"2024-09-12T07:20:52.463032+0000","last_fullsized":"2024-09-12T07:20:52.463032+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:26:17.201737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047491800000000001,"stat_sum":{"num_bytes":35566196,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":21,"num_read":4750,"num_read_kb":22745,"num_write":1325,"num_write_kb":22389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":8633990,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"88'377","reported_seq":4232,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T07:20:51.390850+0000","last_change":"2024-09-12T07:19:44.469508+0000","last_active":"2024-09-12T07:20:51.390850+0000","last_peered":"2024-09-12T07:20:51.390850+0000","last_clean":"2024-09-12T07:20:51.390850+0000","last_became_active":"2024-09-12T07:19:42.450230+0000","last_became_peered":"2024-09-12T07:19:42.450230+0000","last_unstale":"2024-09-12T07:20:51.390850+0000","last_undegraded":"2024-09-12T07:20:51.390850+0000","last_fullsized":"2024-09-12T07:20:51.390850+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:23.236404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076460899999999997,"stat_sum":{"num_bytes":35040157,"num_objects":55,"num_object_clones":9,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":23,"num_read":6529,"num_read_kb":29508,"num_write":386,"num_write_kb":10459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"91'324","reported_seq":3935,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.550015+0000","last_change":"2024-09-12T07:20:47.377709+0000","last_active":"2024-09-12T07:20:52.550015+0000","last_peered":"2024-09-12T07:20:52.550015+0000","last_clean":"2024-09-12T07:20:52.550015+0000","last_became_active":"2024-09-12T07:20:47.375097+0000","last_became_peered":"2024-09-12T07:20:47.375097+0000","last_unstale":"2024-09-12T07:20:52.550015+0000","last_undegraded":"2024-09-12T07:20:52.550015+0000","last_fullsized":"2024-09-12T07:20:52.550015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:54:01.144650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000680892,"stat_sum":{"num_bytes":46915121,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":25,"num_read":3986,"num_read_kb":42495,"num_write":344,"num_write_kb":23002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":14164436,"num_keys_recovered":27,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"91'297","reported_seq":1898,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.465028+0000","last_change":"2024-09-12T07:20:07.379907+0000","last_active":"2024-09-12T07:20:52.465028+0000","last_peered":"2024-09-12T07:20:52.465028+0000","last_clean":"2024-09-12T07:20:52.465028+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:20:52.465028+0000","last_undegraded":"2024-09-12T07:20:52.465028+0000","last_fullsized":"2024-09-12T07:20:52.465028+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:19.998405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054644100000000005,"stat_sum":{"num_bytes":48524526,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":23,"num_read":2924,"num_read_kb":16452,"num_write":291,"num_write_kb":15417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"91'322","reported_seq":9001,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397697+0000","last_change":"2024-09-12T07:20:07.249347+0000","last_active":"2024-09-12T07:20:52.397697+0000","last_peered":"2024-09-12T07:20:52.397697+0000","last_clean":"2024-09-12T07:20:52.397697+0000","last_became_active":"2024-09-12T07:20:07.183065+0000","last_became_peered":"2024-09-12T07:20:07.183065+0000","last_unstale":"2024-09-12T07:20:52.397697+0000","last_undegraded":"2024-09-12T07:20:52.397697+0000","last_fullsized":"2024-09-12T07:20:52.397697+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:51.567398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050446300000000003,"stat_sum":{"num_bytes":52241991,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":16,"num_read":10446,"num_read_kb":49137,"num_write":293,"num_write_kb":15795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":8578058,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"91'211","reported_seq":1874,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505496+0000","last_change":"2024-09-12T07:19:44.482470+0000","last_active":"2024-09-12T07:20:52.505496+0000","last_peered":"2024-09-12T07:20:52.505496+0000","last_clean":"2024-09-12T07:20:52.505496+0000","last_became_active":"2024-09-12T07:19:42.450577+0000","last_became_peered":"2024-09-12T07:19:42.450577+0000","last_unstale":"2024-09-12T07:20:52.505496+0000","last_undegraded":"2024-09-12T07:20:52.505496+0000","last_fullsized":"2024-09-12T07:20:52.505496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:26:36.110003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317836,"stat_sum":{"num_bytes":39092253,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":21,"num_read":1950,"num_read_kb":19995,"num_write":235,"num_write_kb":10818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"90'380","reported_seq":4101,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505529+0000","last_change":"2024-09-12T07:19:44.481850+0000","last_active":"2024-09-12T07:20:52.505529+0000","last_peered":"2024-09-12T07:20:52.505529+0000","last_clean":"2024-09-12T07:20:52.505529+0000","last_became_active":"2024-09-12T07:19:42.452236+0000","last_became_peered":"2024-09-12T07:19:42.452236+0000","last_unstale":"2024-09-12T07:20:52.505529+0000","last_undegraded":"2024-09-12T07:20:52.505529+0000","last_fullsized":"2024-09-12T07:20:52.505529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:29:18.645276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012939710000000001,"stat_sum":{"num_bytes":39322702,"num_objects":58,"num_object_clones":19,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":27,"num_read":5481,"num_read_kb":28298,"num_write":474,"num_write_kb":12384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"89'384","reported_seq":1513,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505487+0000","last_change":"2024-09-12T07:19:44.482179+0000","last_active":"2024-09-12T07:20:52.505487+0000","last_peered":"2024-09-12T07:20:52.505487+0000","last_clean":"2024-09-12T07:20:52.505487+0000","last_became_active":"2024-09-12T07:19:42.452155+0000","last_became_peered":"2024-09-12T07:19:42.452155+0000","last_unstale":"2024-09-12T07:20:52.505487+0000","last_undegraded":"2024-09-12T07:20:52.505487+0000","last_fullsized":"2024-09-12T07:20:52.505487+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:17.679427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011884759999999999,"stat_sum":{"num_bytes":29997006,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":18,"num_read":2292,"num_read_kb":11748,"num_write":469,"num_write_kb":15347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"91'221","reported_seq":1184,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.460050+0000","last_change":"2024-09-12T07:20:37.330497+0000","last_active":"2024-09-12T07:20:52.460050+0000","last_peered":"2024-09-12T07:20:52.460050+0000","last_clean":"2024-09-12T07:20:52.460050+0000","last_became_active":"2024-09-12T07:20:37.328982+0000","last_became_peered":"2024-09-12T07:20:37.328982+0000","last_unstale":"2024-09-12T07:20:52.460050+0000","last_undegraded":"2024-09-12T07:20:52.460050+0000","last_fullsized":"2024-09-12T07:20:52.460050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:53.950746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073032100000000003,"stat_sum":{"num_bytes":62792458,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":1170,"num_read_kb":23205,"num_write":243,"num_write_kb":16624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"90'218","reported_seq":3088,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505316+0000","last_change":"2024-09-12T07:19:44.481275+0000","last_active":"2024-09-12T07:20:52.505316+0000","last_peered":"2024-09-12T07:20:52.505316+0000","last_clean":"2024-09-12T07:20:52.505316+0000","last_became_active":"2024-09-12T07:19:42.452104+0000","last_became_peered":"2024-09-12T07:19:42.452104+0000","last_unstale":"2024-09-12T07:20:52.505316+0000","last_undegraded":"2024-09-12T07:20:52.505316+0000","last_fullsized":"2024-09-12T07:20:52.505316+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:23:47.134372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079083699999999996,"stat_sum":{"num_bytes":22840029,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":3219,"num_read_kb":34127,"num_write":203,"num_write_kb":16207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"90'207","reported_seq":2535,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.395478+0000","last_change":"2024-09-12T07:19:44.456086+0000","last_active":"2024-09-12T07:20:52.395478+0000","last_peered":"2024-09-12T07:20:52.395478+0000","last_clean":"2024-09-12T07:20:52.395478+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:20:52.395478+0000","last_undegraded":"2024-09-12T07:20:52.395478+0000","last_fullsized":"2024-09-12T07:20:52.395478+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:01.686363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060746,"stat_sum":{"num_bytes":33597399,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":3341,"num_read_kb":22721,"num_write":247,"num_write_kb":11637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"90'359","reported_seq":18790,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505348+0000","last_change":"2024-09-12T07:19:44.482564+0000","last_active":"2024-09-12T07:20:52.505348+0000","last_peered":"2024-09-12T07:20:52.505348+0000","last_clean":"2024-09-12T07:20:52.505348+0000","last_became_active":"2024-09-12T07:19:42.456127+0000","last_became_peered":"2024-09-12T07:19:42.456127+0000","last_unstale":"2024-09-12T07:20:52.505348+0000","last_undegraded":"2024-09-12T07:20:52.505348+0000","last_fullsized":"2024-09-12T07:20:52.505348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:42:39.818004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793039999999999,"stat_sum":{"num_bytes":69124901,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":24,"num_read":18890,"num_read_kb":80661,"num_write":467,"num_write_kb":41147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1289771554,"num_objects":1514,"num_object_clones":367,"num_object_copies":3028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1514,"num_whiteouts":630,"num_read":190388,"num_read_kb":926033,"num_write":13898,"num_write_kb":493326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":64106948,"num_keys_recovered":75,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":653631488,"data_stored":793189781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11239,"ondisk_log_size":11239,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":151904,"kb_used_data":112388,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94219936,"statfs":{"total":96636764160,"available":96481214464,"internally_reserved":0,"allocated":115085312,"data_stored":138210280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542165,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55188,"kb_used_data":24004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316652,"statfs":{"total":96636764160,"available":96580251648,"internally_reserved":0,"allocated":24580096,"data_stored":23956063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":128132,"kb_used_data":88628,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94243708,"statfs":{"total":96636764160,"available":96505556992,"internally_reserved":0,"allocated":90755072,"data_stored":106737684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":191352,"kb_used_data":164320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94180488,"statfs":{"total":96636764160,"available":96440819712,"internally_reserved":0,"allocated":168263680,"data_stored":216390839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542165,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148272,"kb_used_data":99036,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94223568,"statfs":{"total":96636764160,"available":96484933632,"internally_reserved":0,"allocated":101412864,"data_stored":118182227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50104,"kb_used_data":14756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321736,"statfs":{"total":96636764160,"available":96585457664,"internally_reserved":0,"allocated":15110144,"data_stored":14698545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80220,"kb_used_data":53188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94291620,"statfs":{"total":96636764160,"available":96554618880,"internally_reserved":0,"allocated":54464512,"data_stored":68666701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542165,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117788,"kb_used_data":90756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94254052,"statfs":{"total":96636764160,"available":96516149248,"internally_reserved":0,"allocated":92934144,"data_stored":110804336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":91287552,"data_stored":109730779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53551104,"data_stored":68301979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14049280,"data_stored":14211392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":100560896,"data_stored":117876765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":167190528,"data_stored":215897844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":89915392,"data_stored":106459439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23572480,"data_stored":23519217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":113504256,"data_stored":137192366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:20:56.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:20:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:5103 write 0x8c1277e thru 0x8c2266e (0xfef1 bytes) 2024-09-12T07:20:56.861 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:56.861 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:56.861 INFO:teuthology.orchestra.run.smithi046.stdout:5104 read 0xff519c thru 0x10036d9 (0xe53e bytes) 2024-09-12T07:20:56.861 INFO:teuthology.orchestra.run.smithi046.stdout:5105 write 0xba9b7c thru 0xbb64be (0xc943 bytes) 2024-09-12T07:20:56.869 INFO:teuthology.orchestra.run.smithi046.stdout:5107 write 0x98c08d8 thru 0x98c212c (0x1855 bytes) 2024-09-12T07:20:56.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:20:56.872 INFO:teuthology.orchestra.run.smithi046.stdout:5108 write 0x8e44440 thru 0x8e47fd8 (0x3b99 bytes) 2024-09-12T07:20:56.874 INFO:teuthology.orchestra.run.smithi046.stdout:5109 write 0x8e605d6 thru 0x8e67539 (0x6f64 bytes) 2024-09-12T07:20:56.875 INFO:teuthology.orchestra.run.smithi046.stdout:5110 write 0xd908d1 thru 0xda05cf (0xfcff bytes) 2024-09-12T07:20:56.878 INFO:teuthology.orchestra.run.smithi046.stdout:5111 write 0x8b82bc8 thru 0x8b8f122 (0xc55b bytes) 2024-09-12T07:20:56.886 INFO:teuthology.orchestra.run.smithi046.stdout:5114 write 0xa6fef6d thru 0xa70d1f6 (0xe28a bytes) 2024-09-12T07:20:56.892 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:56.894+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:56.894 INFO:teuthology.orchestra.run.smithi046.stdout:5115 read 0x6e69bb1 thru 0x6e6dd3d (0x418d bytes) 2024-09-12T07:20:56.894 INFO:teuthology.orchestra.run.smithi046.stdout:5116 read 0x9b3a91b thru 0x9b48b1d (0xe203 bytes) 2024-09-12T07:20:56.901 INFO:teuthology.orchestra.run.smithi046.stdout:5118 writesame 0xb3a5daf thru 0xb3ad614 data_size 0x21(0x7866 bytes) 2024-09-12T07:20:56.901 INFO:teuthology.orchestra.run.smithi046.stdout:5119 punch from 0xd325a5 to 0xd34b50, (0x25ab bytes) 2024-09-12T07:20:56.901 INFO:teuthology.orchestra.run.smithi046.stdout:5120 write 0x9bb5ebd thru 0x9bbde71 (0x7fb5 bytes) 2024-09-12T07:20:56.902 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.903 INFO:teuthology.orchestra.run.smithi046.stdout:5122 trunc from 0xb3ad615 to 0xa72f3e9 2024-09-12T07:20:56.910 INFO:teuthology.orchestra.run.smithi046.stdout:5125 read 0x95387d4 thru 0x9547fda (0xf807 bytes) 2024-09-12T07:20:56.916 INFO:teuthology.orchestra.run.smithi046.stdout:5126 write 0xa905495 thru 0xa906f9c (0x1b08 bytes) 2024-09-12T07:20:56.918 INFO:teuthology.orchestra.run.smithi046.stdout:5127 write 0x7be29ae thru 0x7be4b03 (0x2156 bytes) 2024-09-12T07:20:56.920 INFO:teuthology.orchestra.run.smithi046.stdout:5128 write 0x4336770 thru 0x433c1fd (0x5a8e bytes) 2024-09-12T07:20:56.925 INFO:teuthology.orchestra.run.smithi046.stdout:5130 trunc from 0xa906f9d to 0x9d02574 2024-09-12T07:20:56.936 INFO:teuthology.orchestra.run.smithi046.stdout:5132 read 0x7c41a64 thru 0x7c4c9af (0xaf4c bytes) 2024-09-12T07:20:56.945 INFO:teuthology.orchestra.run.smithi046.stdout:5133 writesame 0xc4c88a1 thru 0xc4d08a4 data_size 0x6(0x8004 bytes) 2024-09-12T07:20:56.945 INFO:teuthology.orchestra.run.smithi046.stdout:5134 writesame 0x6cf7bfd thru 0x6cffc48 data_size 0x54(0x804c bytes) 2024-09-12T07:20:56.956 INFO:teuthology.orchestra.run.smithi046.stdout:5137 write 0xd09af7d thru 0xd0aa317 (0xf39b bytes) 2024-09-12T07:20:56.961 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.961 INFO:teuthology.orchestra.run.smithi046.stdout:5139 punch from 0x10f8b12 to 0x1103ce0, (0xb1ce bytes) 2024-09-12T07:20:56.961 INFO:teuthology.orchestra.run.smithi046.stdout:5140 read 0x8c74dfc thru 0x8c828bf (0xdac4 bytes) 2024-09-12T07:20:56.961 INFO:teuthology.orchestra.run.smithi046.stdout:5142 writesame 0x199d7ce thru 0x19abdb1 data_size 0x2(0xe5e4 bytes) 2024-09-12T07:20:56.975 INFO:teuthology.orchestra.run.smithi046.stdout:5143 read 0xad7582d thru 0xad7d063 (0x7837 bytes) 2024-09-12T07:20:56.975 INFO:teuthology.orchestra.run.smithi046.stdout:5144 writesame 0x7ba356f thru 0x7bb24ec data_size 0x5(0xef7e bytes) 2024-09-12T07:20:56.980 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:56.980 INFO:teuthology.orchestra.run.smithi046.stdout:5147 write 0x682e1af thru 0x683160c (0x345e bytes) 2024-09-12T07:20:56.982 INFO:teuthology.orchestra.run.smithi046.stdout:5148 write 0xacd0e5f thru 0xacd6347 (0x54e9 bytes) 2024-09-12T07:20:56.994 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:56.995+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:57.002 INFO:teuthology.orchestra.run.smithi046.stdout:5152 trunc from 0xd0aa318 to 0x1f8d84 2024-09-12T07:20:57.002 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:57.002 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:57.014 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:20:57.033 INFO:teuthology.orchestra.run.smithi046.stdout:5153 write 0x890da9a thru 0x890e02b (0x592 bytes) 2024-09-12T07:20:57.036 INFO:teuthology.orchestra.run.smithi046.stdout:5154 read 0x34f78b2 thru 0x34f7955 (0xa4 bytes) 2024-09-12T07:20:57.036 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:57.036 INFO:teuthology.orchestra.run.smithi046.stdout:5157 read 0x34e97ad thru 0x34f2b0a (0x935e bytes) 2024-09-12T07:20:57.036 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:57.043 INFO:teuthology.orchestra.run.smithi046.stdout:5159 trunc from 0x890e02c to 0x2690dbb 2024-09-12T07:20:57.072 INFO:teuthology.orchestra.run.smithi046.stdout:5160 write 0xe1bb8bd thru 0xe1c906a (0xd7ae bytes) 2024-09-12T07:20:57.077 INFO:teuthology.orchestra.run.smithi046.stdout:5161 writesame 0x61b9914 thru 0x61c1bf3 data_size 0x30(0x82e0 bytes) 2024-09-12T07:20:57.081 INFO:teuthology.orchestra.run.smithi046.stdout:5162 read 0xd6d9f9b thru 0xd6e6623 (0xc689 bytes) 2024-09-12T07:20:57.081 INFO:teuthology.orchestra.run.smithi046.stdout:5163 write 0xa86bd09 thru 0xa86fef2 (0x41ea bytes) 2024-09-12T07:20:57.083 INFO:teuthology.orchestra.run.smithi046.stdout:5164 read 0x6edaa6e thru 0x6edf71a (0x4cad bytes) 2024-09-12T07:20:57.084 INFO:teuthology.orchestra.run.smithi046.stdout:5167 trunc from 0xe1c906b to 0xdd845fa 2024-09-12T07:20:57.094 INFO:teuthology.orchestra.run.smithi046.stdout:5168 trunc from 0xdd845fa to 0xb347afe 2024-09-12T07:20:57.094 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:57.096+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:57.111 INFO:teuthology.orchestra.run.smithi046.stdout:5169 writesame 0xcc32e50 thru 0xcc35b81 data_size 0xd(0x2d32 bytes) 2024-09-12T07:20:57.111 INFO:teuthology.orchestra.run.smithi046.stdout:5171 write 0x7ac8eb6 thru 0x7ad0eec (0x8037 bytes) 2024-09-12T07:20:57.115 INFO:teuthology.orchestra.run.smithi046.stdout:5173 read 0xa5bce75 thru 0xa5c4e81 (0x800d bytes) 2024-09-12T07:20:57.122 INFO:teuthology.orchestra.run.smithi046.stdout:5175 write 0xe00f327 thru 0xe01b884 (0xc55e bytes) 2024-09-12T07:20:57.127 INFO:teuthology.orchestra.run.smithi046.stdout:5176 read 0xc47faee thru 0xc485e99 (0x63ac bytes) 2024-09-12T07:20:57.127 INFO:teuthology.orchestra.run.smithi046.stdout:5178 write 0x2f7f00c thru 0x2f86e0c (0x7e01 bytes) 2024-09-12T07:20:57.131 INFO:teuthology.orchestra.run.smithi046.stdout:5179 writesame 0x3afe681 thru 0x3b0031c data_size 0x2(0x1c9c bytes) 2024-09-12T07:20:57.135 INFO:teuthology.orchestra.run.smithi046.stdout:5180 read 0xb69d5f5 thru 0xb6a1cb0 (0x46bc bytes) 2024-09-12T07:20:57.138 INFO:teuthology.orchestra.run.smithi046.stdout:5181 trunc from 0xe01b885 to 0xb37287e 2024-09-12T07:20:57.159 INFO:teuthology.orchestra.run.smithi046.stdout:5183 write 0xccc2bca thru 0xccccab0 (0x9ee7 bytes) 2024-09-12T07:20:57.162 INFO:teuthology.orchestra.run.smithi046.stdout:5184 punch from 0x42b83b1 to 0x42b9f88, (0x1bd7 bytes) 2024-09-12T07:20:57.163 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:57.163 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:57.164 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:57.164 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:57.172 INFO:teuthology.orchestra.run.smithi046.stdout:5187 trunc from 0xccccab1 to 0x4bfb455 2024-09-12T07:20:57.179 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:20:57.181 INFO:teuthology.orchestra.run.smithi046.stdout:5190 read 0x4a30d2c thru 0x4a403c3 (0xf698 bytes) 2024-09-12T07:20:57.184 INFO:teuthology.orchestra.run.smithi046.stdout:5191 trunc from 0x4bfb455 to 0x70a48b8 2024-09-12T07:20:57.190 INFO:teuthology.orchestra.run.smithi046.stdout:5192 read 0x31f1fd6 thru 0x31f4417 (0x2442 bytes) 2024-09-12T07:20:57.190 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:57.190 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:57.190 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:57.190 INFO:teuthology.orchestra.run.smithi046.stdout:5197 read 0x6834f9e thru 0x6841d09 (0xcd6c bytes) 2024-09-12T07:20:57.190 INFO:teuthology.orchestra.run.smithi046.stdout:5198 punch from 0xaa216b to 0xaa5333, (0x31c8 bytes) 2024-09-12T07:20:57.190 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:57.195 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:57.196+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:57.202 INFO:teuthology.orchestra.run.smithi046.stdout:5201 write 0xd057dd7 thru 0xd0664fb (0xe725 bytes) 2024-09-12T07:20:57.208 INFO:teuthology.orchestra.run.smithi046.stdout:5202 trunc from 0xd0664fc to 0xdbd1b9d 2024-09-12T07:20:57.214 INFO:teuthology.orchestra.run.smithi046.stdout:5206 read 0x52d37ff thru 0x52d8812 (0x5014 bytes) 2024-09-12T07:20:57.214 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:57.214 INFO:teuthology.orchestra.run.smithi046.stdout:5210 writesame 0x30e5333 thru 0x30f35bd data_size 0x7(0xe28b bytes) 2024-09-12T07:20:57.219 INFO:teuthology.orchestra.run.smithi046.stdout:5212 write 0xbe5eb4d thru 0xbe6d75b (0xec0f bytes) 2024-09-12T07:20:57.224 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:20:57.225 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:20:57.227 INFO:teuthology.orchestra.run.smithi046.stdout:5213 trunc from 0xdbd1b9d to 0xd3f2e0d 2024-09-12T07:20:57.234 INFO:teuthology.orchestra.run.smithi046.stdout:5216 punch from 0x1f65691 to 0x1f65c7a, (0x5e9 bytes) 2024-09-12T07:20:57.234 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:57.234 INFO:teuthology.orchestra.run.smithi046.stdout:5218 write 0x5241fff thru 0x524f314 (0xd316 bytes) 2024-09-12T07:20:57.238 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-12T07:20:56.420130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1290361941,"num_objects":1518,"num_object_clones":367,"num_object_copies":3036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1518,"num_whiteouts":630,"num_read":190454,"num_read_kb":926088,"num_write":14001,"num_write_kb":494956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":64106948,"num_keys_recovered":75,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11314,"ondisk_log_size":11314,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":922960,"kb_used_data":647076,"kb_used_omap":12,"kb_used_meta":275699,"kb_avail":754051760,"statfs":{"total":773094113280,"available":772149002240,"internally_reserved":0,"allocated":662605824,"data_stored":797646675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":282316337},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":88302728,"num_objects":57,"num_object_clones":4,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":5419,"num_read_kb":12237,"num_write":852,"num_write_kb":9065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023785"},"pg_stats":[{"pgid":"3.1f","version":"91'446","reported_seq":2323,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.486579+0000","last_change":"2024-09-12T07:19:44.455827+0000","last_active":"2024-09-12T07:20:52.486579+0000","last_peered":"2024-09-12T07:20:52.486579+0000","last_clean":"2024-09-12T07:20:52.486579+0000","last_became_active":"2024-09-12T07:19:42.451212+0000","last_became_peered":"2024-09-12T07:19:42.451212+0000","last_unstale":"2024-09-12T07:20:52.486579+0000","last_undegraded":"2024-09-12T07:20:52.486579+0000","last_fullsized":"2024-09-12T07:20:52.486579+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:03:26.073069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001023672,"stat_sum":{"num_bytes":13840385,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":3701,"num_read_kb":16225,"num_write":620,"num_write_kb":6362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"90'292","reported_seq":1501,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.463172+0000","last_change":"2024-09-12T07:20:07.305432+0000","last_active":"2024-09-12T07:20:52.463172+0000","last_peered":"2024-09-12T07:20:52.463172+0000","last_clean":"2024-09-12T07:20:52.463172+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:20:52.463172+0000","last_undegraded":"2024-09-12T07:20:52.463172+0000","last_fullsized":"2024-09-12T07:20:52.463172+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:23.432336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081935099999999996,"stat_sum":{"num_bytes":58732364,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":18969,"num_read_kb":35160,"num_write":270,"num_write_kb":18916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"91'333","reported_seq":18517,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.553376+0000","last_change":"2024-09-12T07:19:44.464376+0000","last_active":"2024-09-12T07:20:52.553376+0000","last_peered":"2024-09-12T07:20:52.553376+0000","last_clean":"2024-09-12T07:20:52.553376+0000","last_became_active":"2024-09-12T07:19:42.451973+0000","last_became_peered":"2024-09-12T07:19:42.451973+0000","last_unstale":"2024-09-12T07:20:52.553376+0000","last_undegraded":"2024-09-12T07:20:52.553376+0000","last_fullsized":"2024-09-12T07:20:52.553376+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:14.774992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036949999999999,"stat_sum":{"num_bytes":41834087,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":22,"num_read":18658,"num_read_kb":31056,"num_write":384,"num_write_kb":12762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"90'440","reported_seq":2792,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397601+0000","last_change":"2024-09-12T07:19:44.464287+0000","last_active":"2024-09-12T07:20:52.397601+0000","last_peered":"2024-09-12T07:20:52.397601+0000","last_clean":"2024-09-12T07:20:52.397601+0000","last_became_active":"2024-09-12T07:19:42.455338+0000","last_became_peered":"2024-09-12T07:19:42.455338+0000","last_unstale":"2024-09-12T07:20:52.397601+0000","last_undegraded":"2024-09-12T07:20:52.397601+0000","last_fullsized":"2024-09-12T07:20:52.397601+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:21.916787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012347180000000001,"stat_sum":{"num_bytes":43084583,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":3357,"num_read_kb":15811,"num_write":600,"num_write_kb":8810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"91'305","reported_seq":17822,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.437869+0000","last_change":"2024-09-12T07:19:44.456652+0000","last_active":"2024-09-12T07:20:52.437869+0000","last_peered":"2024-09-12T07:20:52.437869+0000","last_clean":"2024-09-12T07:20:52.437869+0000","last_became_active":"2024-09-12T07:19:42.454990+0000","last_became_peered":"2024-09-12T07:19:42.454990+0000","last_unstale":"2024-09-12T07:20:52.437869+0000","last_undegraded":"2024-09-12T07:20:52.437869+0000","last_fullsized":"2024-09-12T07:20:52.437869+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:54.109441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049866,"stat_sum":{"num_bytes":44552147,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":18296,"num_read_kb":47346,"num_write":318,"num_write_kb":15496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"91'296","reported_seq":3196,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.544527+0000","last_change":"2024-09-12T07:19:44.459417+0000","last_active":"2024-09-12T07:20:52.544527+0000","last_peered":"2024-09-12T07:20:52.544527+0000","last_clean":"2024-09-12T07:20:52.544527+0000","last_became_active":"2024-09-12T07:19:42.454620+0000","last_became_peered":"2024-09-12T07:19:42.454620+0000","last_unstale":"2024-09-12T07:20:52.544527+0000","last_undegraded":"2024-09-12T07:20:52.544527+0000","last_fullsized":"2024-09-12T07:20:52.544527+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:36.284865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090501500000000001,"stat_sum":{"num_bytes":39289879,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":23,"num_read":3728,"num_read_kb":12271,"num_write":277,"num_write_kb":17032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"91'323","reported_seq":5006,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397461+0000","last_change":"2024-09-12T07:20:10.196744+0000","last_active":"2024-09-12T07:20:52.397461+0000","last_peered":"2024-09-12T07:20:52.397461+0000","last_clean":"2024-09-12T07:20:52.397461+0000","last_became_active":"2024-09-12T07:20:10.195312+0000","last_became_peered":"2024-09-12T07:20:10.195312+0000","last_unstale":"2024-09-12T07:20:52.397461+0000","last_undegraded":"2024-09-12T07:20:52.397461+0000","last_fullsized":"2024-09-12T07:20:52.397461+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:30:28.696660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113156,"stat_sum":{"num_bytes":34739920,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":15,"num_read":5309,"num_read_kb":38951,"num_write":425,"num_write_kb":12939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1101525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"91'473","reported_seq":2867,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.503709+0000","last_change":"2024-09-12T07:20:07.446518+0000","last_active":"2024-09-12T07:20:52.503709+0000","last_peered":"2024-09-12T07:20:52.503709+0000","last_clean":"2024-09-12T07:20:52.503709+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:20:52.503709+0000","last_undegraded":"2024-09-12T07:20:52.503709+0000","last_fullsized":"2024-09-12T07:20:52.503709+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:37.690882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000443294,"stat_sum":{"num_bytes":54337354,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":13,"num_read":5513,"num_read_kb":41321,"num_write":642,"num_write_kb":18029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"91'338","reported_seq":4975,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.532129+0000","last_change":"2024-09-12T07:19:44.456115+0000","last_active":"2024-09-12T07:20:52.532129+0000","last_peered":"2024-09-12T07:20:52.532129+0000","last_clean":"2024-09-12T07:20:52.532129+0000","last_became_active":"2024-09-12T07:19:42.452617+0000","last_became_peered":"2024-09-12T07:19:42.452617+0000","last_unstale":"2024-09-12T07:20:52.532129+0000","last_undegraded":"2024-09-12T07:20:52.532129+0000","last_fullsized":"2024-09-12T07:20:52.532129+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:29:53.445014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077251,"stat_sum":{"num_bytes":41864668,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":16,"num_read":5916,"num_read_kb":17020,"num_write":340,"num_write_kb":22272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"90'158","reported_seq":878,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397648+0000","last_change":"2024-09-12T07:19:44.463507+0000","last_active":"2024-09-12T07:20:52.397648+0000","last_peered":"2024-09-12T07:20:52.397648+0000","last_clean":"2024-09-12T07:20:52.397648+0000","last_became_active":"2024-09-12T07:19:42.455210+0000","last_became_peered":"2024-09-12T07:19:42.455210+0000","last_unstale":"2024-09-12T07:20:52.397648+0000","last_undegraded":"2024-09-12T07:20:52.397648+0000","last_fullsized":"2024-09-12T07:20:52.397648+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:10.840067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049753700000000004,"stat_sum":{"num_bytes":18453957,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":23,"num_read":616,"num_read_kb":7901,"num_write":161,"num_write_kb":6863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'192","reported_seq":6212,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505400+0000","last_change":"2024-09-12T07:20:07.261139+0000","last_active":"2024-09-12T07:20:52.505400+0000","last_peered":"2024-09-12T07:20:52.505400+0000","last_clean":"2024-09-12T07:20:52.505400+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:20:52.505400+0000","last_undegraded":"2024-09-12T07:20:52.505400+0000","last_fullsized":"2024-09-12T07:20:52.505400+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.425867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012225490000000001,"stat_sum":{"num_bytes":22445643,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":6416,"num_read_kb":43219,"num_write":167,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"89'285","reported_seq":4915,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505355+0000","last_change":"2024-09-12T07:19:44.481796+0000","last_active":"2024-09-12T07:20:52.505355+0000","last_peered":"2024-09-12T07:20:52.505355+0000","last_clean":"2024-09-12T07:20:52.505355+0000","last_became_active":"2024-09-12T07:19:42.452277+0000","last_became_peered":"2024-09-12T07:19:42.452277+0000","last_unstale":"2024-09-12T07:20:52.505355+0000","last_undegraded":"2024-09-12T07:20:52.505355+0000","last_fullsized":"2024-09-12T07:20:52.505355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:01.375693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012485160000000001,"stat_sum":{"num_bytes":57888336,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":19,"num_read":5159,"num_read_kb":60603,"num_write":313,"num_write_kb":30134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505578+0000","last_change":"2024-09-12T07:20:37.329418+0000","last_active":"2024-09-12T07:20:52.505578+0000","last_peered":"2024-09-12T07:20:52.505578+0000","last_clean":"2024-09-12T07:20:52.505578+0000","last_became_active":"2024-09-12T07:20:37.329145+0000","last_became_peered":"2024-09-12T07:20:37.329145+0000","last_unstale":"2024-09-12T07:20:52.505578+0000","last_undegraded":"2024-09-12T07:20:52.505578+0000","last_fullsized":"2024-09-12T07:20:52.505578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:58.555569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"89'429","reported_seq":1717,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.395525+0000","last_change":"2024-09-12T07:19:44.455257+0000","last_active":"2024-09-12T07:20:52.395525+0000","last_peered":"2024-09-12T07:20:52.395525+0000","last_clean":"2024-09-12T07:20:52.395525+0000","last_became_active":"2024-09-12T07:19:42.449730+0000","last_became_peered":"2024-09-12T07:19:42.449730+0000","last_unstale":"2024-09-12T07:20:52.395525+0000","last_undegraded":"2024-09-12T07:20:52.395525+0000","last_fullsized":"2024-09-12T07:20:52.395525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:16.258754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059259600000000003,"stat_sum":{"num_bytes":20649904,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":22,"num_read":2641,"num_read_kb":10111,"num_write":561,"num_write_kb":8756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397643+0000","last_change":"2024-09-12T07:19:32.228066+0000","last_active":"2024-09-12T07:20:52.397643+0000","last_peered":"2024-09-12T07:20:52.397643+0000","last_clean":"2024-09-12T07:20:52.397643+0000","last_became_active":"2024-09-12T07:19:30.219950+0000","last_became_peered":"2024-09-12T07:19:30.219950+0000","last_unstale":"2024-09-12T07:20:52.397643+0000","last_undegraded":"2024-09-12T07:20:52.397643+0000","last_fullsized":"2024-09-12T07:20:52.397643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:21.123510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059695500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"91'360","reported_seq":1420,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.558392+0000","last_change":"2024-09-12T07:20:07.211023+0000","last_active":"2024-09-12T07:20:52.558392+0000","last_peered":"2024-09-12T07:20:52.558392+0000","last_clean":"2024-09-12T07:20:52.558392+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:20:52.558392+0000","last_undegraded":"2024-09-12T07:20:52.558392+0000","last_fullsized":"2024-09-12T07:20:52.558392+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:05.723251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087343100000000003,"stat_sum":{"num_bytes":44035048,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":1810,"num_read_kb":13214,"num_write":443,"num_write_kb":7845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505517+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:20:52.505517+0000","last_peered":"2024-09-12T07:20:52.505517+0000","last_clean":"2024-09-12T07:20:52.505517+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:20:52.505517+0000","last_undegraded":"2024-09-12T07:20:52.505517+0000","last_fullsized":"2024-09-12T07:20:52.505517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"88'340","reported_seq":1872,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397480+0000","last_change":"2024-09-12T07:19:44.458925+0000","last_active":"2024-09-12T07:20:52.397480+0000","last_peered":"2024-09-12T07:20:52.397480+0000","last_clean":"2024-09-12T07:20:52.397480+0000","last_became_active":"2024-09-12T07:19:42.452704+0000","last_became_peered":"2024-09-12T07:19:42.452704+0000","last_unstale":"2024-09-12T07:20:52.397480+0000","last_undegraded":"2024-09-12T07:20:52.397480+0000","last_fullsized":"2024-09-12T07:20:52.397480+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:17:15.336931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045577200000000001,"stat_sum":{"num_bytes":31139649,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":3043,"num_read_kb":10221,"num_write":441,"num_write_kb":6478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397540+0000","last_change":"2024-09-12T07:19:32.228114+0000","last_active":"2024-09-12T07:20:52.397540+0000","last_peered":"2024-09-12T07:20:52.397540+0000","last_clean":"2024-09-12T07:20:52.397540+0000","last_became_active":"2024-09-12T07:19:30.718766+0000","last_became_peered":"2024-09-12T07:19:30.718766+0000","last_unstale":"2024-09-12T07:20:52.397540+0000","last_undegraded":"2024-09-12T07:20:52.397540+0000","last_fullsized":"2024-09-12T07:20:52.397540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.116214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000620441,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"90'270","reported_seq":4678,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397767+0000","last_change":"2024-09-12T07:20:07.289858+0000","last_active":"2024-09-12T07:20:52.397767+0000","last_peered":"2024-09-12T07:20:52.397767+0000","last_clean":"2024-09-12T07:20:52.397767+0000","last_became_active":"2024-09-12T07:20:07.183819+0000","last_became_peered":"2024-09-12T07:20:07.183819+0000","last_unstale":"2024-09-12T07:20:52.397767+0000","last_undegraded":"2024-09-12T07:20:52.397767+0000","last_fullsized":"2024-09-12T07:20:52.397767+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:41:55.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015134619999999999,"stat_sum":{"num_bytes":59078065,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":23,"num_read":5060,"num_read_kb":50291,"num_write":347,"num_write_kb":20933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":3070354,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"91'200","reported_seq":2066,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.484934+0000","last_change":"2024-09-12T07:19:44.456465+0000","last_active":"2024-09-12T07:20:52.484934+0000","last_peered":"2024-09-12T07:20:52.484934+0000","last_clean":"2024-09-12T07:20:52.484934+0000","last_became_active":"2024-09-12T07:19:42.454103+0000","last_became_peered":"2024-09-12T07:19:42.454103+0000","last_unstale":"2024-09-12T07:20:52.484934+0000","last_undegraded":"2024-09-12T07:20:52.484934+0000","last_fullsized":"2024-09-12T07:20:52.484934+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:01:25.840939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010980880000000001,"stat_sum":{"num_bytes":23845044,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":26,"num_read":1747,"num_read_kb":7307,"num_write":182,"num_write_kb":8837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":160,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397578+0000","last_change":"2024-09-12T07:19:32.226380+0000","last_active":"2024-09-12T07:20:52.397578+0000","last_peered":"2024-09-12T07:20:52.397578+0000","last_clean":"2024-09-12T07:20:52.397578+0000","last_became_active":"2024-09-12T07:19:30.216705+0000","last_became_peered":"2024-09-12T07:19:30.216705+0000","last_unstale":"2024-09-12T07:20:52.397578+0000","last_undegraded":"2024-09-12T07:20:52.397578+0000","last_fullsized":"2024-09-12T07:20:52.397578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:32:18.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"87'545","reported_seq":2475,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397355+0000","last_change":"2024-09-12T07:19:44.458868+0000","last_active":"2024-09-12T07:20:52.397355+0000","last_peered":"2024-09-12T07:20:52.397355+0000","last_clean":"2024-09-12T07:20:52.397355+0000","last_became_active":"2024-09-12T07:19:42.451807+0000","last_became_peered":"2024-09-12T07:19:42.451807+0000","last_unstale":"2024-09-12T07:20:52.397355+0000","last_undegraded":"2024-09-12T07:20:52.397355+0000","last_fullsized":"2024-09-12T07:20:52.397355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:58:02.768787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048363399999999998,"stat_sum":{"num_bytes":30205412,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":26,"num_read":4592,"num_read_kb":10908,"num_write":795,"num_write_kb":10412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T07:20:51.390827+0000","last_change":"2024-09-12T07:20:07.183543+0000","last_active":"2024-09-12T07:20:51.390827+0000","last_peered":"2024-09-12T07:20:51.390827+0000","last_clean":"2024-09-12T07:20:51.390827+0000","last_became_active":"2024-09-12T07:20:07.183213+0000","last_became_peered":"2024-09-12T07:20:07.183213+0000","last_unstale":"2024-09-12T07:20:51.390827+0000","last_undegraded":"2024-09-12T07:20:51.390827+0000","last_fullsized":"2024-09-12T07:20:51.390827+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.634459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"86'225","reported_seq":3380,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.395402+0000","last_change":"2024-09-12T07:19:44.455560+0000","last_active":"2024-09-12T07:20:52.395402+0000","last_peered":"2024-09-12T07:20:52.395402+0000","last_clean":"2024-09-12T07:20:52.395402+0000","last_became_active":"2024-09-12T07:19:42.451970+0000","last_became_peered":"2024-09-12T07:19:42.451970+0000","last_unstale":"2024-09-12T07:20:52.395402+0000","last_undegraded":"2024-09-12T07:20:52.395402+0000","last_fullsized":"2024-09-12T07:20:52.395402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:52.854392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083352100000000004,"stat_sum":{"num_bytes":45285524,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":4593,"num_read_kb":35426,"num_write":234,"num_write_kb":23622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505298+0000","last_change":"2024-09-12T07:19:33.033384+0000","last_active":"2024-09-12T07:20:52.505298+0000","last_peered":"2024-09-12T07:20:52.505298+0000","last_clean":"2024-09-12T07:20:52.505298+0000","last_became_active":"2024-09-12T07:19:30.217430+0000","last_became_peered":"2024-09-12T07:19:30.217430+0000","last_unstale":"2024-09-12T07:20:52.505298+0000","last_undegraded":"2024-09-12T07:20:52.505298+0000","last_fullsized":"2024-09-12T07:20:52.505298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:10.689865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'874","reported_seq":3498,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.463057+0000","last_change":"2024-09-12T07:20:07.467030+0000","last_active":"2024-09-12T07:20:52.463057+0000","last_peered":"2024-09-12T07:20:52.463057+0000","last_clean":"2024-09-12T07:20:52.463057+0000","last_became_active":"2024-09-12T07:20:07.180577+0000","last_became_peered":"2024-09-12T07:20:07.180577+0000","last_unstale":"2024-09-12T07:20:52.463057+0000","last_undegraded":"2024-09-12T07:20:52.463057+0000","last_fullsized":"2024-09-12T07:20:52.463057+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:31:46.435494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089638599999999999,"stat_sum":{"num_bytes":49414846,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":25,"num_read":6286,"num_read_kb":40579,"num_write":1401,"num_write_kb":17418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":261251,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":159,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397589+0000","last_change":"2024-09-12T07:19:32.225708+0000","last_active":"2024-09-12T07:20:52.397589+0000","last_peered":"2024-09-12T07:20:52.397589+0000","last_clean":"2024-09-12T07:20:52.397589+0000","last_became_active":"2024-09-12T07:19:30.216530+0000","last_became_peered":"2024-09-12T07:19:30.216530+0000","last_unstale":"2024-09-12T07:20:52.397589+0000","last_undegraded":"2024-09-12T07:20:52.397589+0000","last_fullsized":"2024-09-12T07:20:52.397589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:38.201729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064059200000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":249,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505312+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:20:52.505312+0000","last_peered":"2024-09-12T07:20:52.505312+0000","last_clean":"2024-09-12T07:20:52.505312+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:20:52.505312+0000","last_undegraded":"2024-09-12T07:20:52.505312+0000","last_fullsized":"2024-09-12T07:20:52.505312+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"90'815","reported_seq":2017,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.463032+0000","last_change":"2024-09-12T07:20:07.428482+0000","last_active":"2024-09-12T07:20:52.463032+0000","last_peered":"2024-09-12T07:20:52.463032+0000","last_clean":"2024-09-12T07:20:52.463032+0000","last_became_active":"2024-09-12T07:20:07.180763+0000","last_became_peered":"2024-09-12T07:20:07.180763+0000","last_unstale":"2024-09-12T07:20:52.463032+0000","last_undegraded":"2024-09-12T07:20:52.463032+0000","last_fullsized":"2024-09-12T07:20:52.463032+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:26:17.201737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047491800000000001,"stat_sum":{"num_bytes":35566196,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":21,"num_read":4750,"num_read_kb":22745,"num_write":1325,"num_write_kb":22389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":8633990,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"88'377","reported_seq":4232,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T07:20:51.390850+0000","last_change":"2024-09-12T07:19:44.469508+0000","last_active":"2024-09-12T07:20:51.390850+0000","last_peered":"2024-09-12T07:20:51.390850+0000","last_clean":"2024-09-12T07:20:51.390850+0000","last_became_active":"2024-09-12T07:19:42.450230+0000","last_became_peered":"2024-09-12T07:19:42.450230+0000","last_unstale":"2024-09-12T07:20:51.390850+0000","last_undegraded":"2024-09-12T07:20:51.390850+0000","last_fullsized":"2024-09-12T07:20:51.390850+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:23.236404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076460899999999997,"stat_sum":{"num_bytes":35040157,"num_objects":55,"num_object_clones":9,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":23,"num_read":6529,"num_read_kb":29508,"num_write":386,"num_write_kb":10459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"91'324","reported_seq":3935,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.550015+0000","last_change":"2024-09-12T07:20:47.377709+0000","last_active":"2024-09-12T07:20:52.550015+0000","last_peered":"2024-09-12T07:20:52.550015+0000","last_clean":"2024-09-12T07:20:52.550015+0000","last_became_active":"2024-09-12T07:20:47.375097+0000","last_became_peered":"2024-09-12T07:20:47.375097+0000","last_unstale":"2024-09-12T07:20:52.550015+0000","last_undegraded":"2024-09-12T07:20:52.550015+0000","last_fullsized":"2024-09-12T07:20:52.550015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:54:01.144650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000680892,"stat_sum":{"num_bytes":46915121,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":25,"num_read":3986,"num_read_kb":42495,"num_write":344,"num_write_kb":23002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":14164436,"num_keys_recovered":27,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"91'297","reported_seq":1898,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.465028+0000","last_change":"2024-09-12T07:20:07.379907+0000","last_active":"2024-09-12T07:20:52.465028+0000","last_peered":"2024-09-12T07:20:52.465028+0000","last_clean":"2024-09-12T07:20:52.465028+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:20:52.465028+0000","last_undegraded":"2024-09-12T07:20:52.465028+0000","last_fullsized":"2024-09-12T07:20:52.465028+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:19.998405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054644100000000005,"stat_sum":{"num_bytes":48524526,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":23,"num_read":2924,"num_read_kb":16452,"num_write":291,"num_write_kb":15417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"91'322","reported_seq":9001,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.397697+0000","last_change":"2024-09-12T07:20:07.249347+0000","last_active":"2024-09-12T07:20:52.397697+0000","last_peered":"2024-09-12T07:20:52.397697+0000","last_clean":"2024-09-12T07:20:52.397697+0000","last_became_active":"2024-09-12T07:20:07.183065+0000","last_became_peered":"2024-09-12T07:20:07.183065+0000","last_unstale":"2024-09-12T07:20:52.397697+0000","last_undegraded":"2024-09-12T07:20:52.397697+0000","last_fullsized":"2024-09-12T07:20:52.397697+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:51.567398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050446300000000003,"stat_sum":{"num_bytes":52241991,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":16,"num_read":10446,"num_read_kb":49137,"num_write":293,"num_write_kb":15795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":8578058,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"91'211","reported_seq":1874,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505496+0000","last_change":"2024-09-12T07:19:44.482470+0000","last_active":"2024-09-12T07:20:52.505496+0000","last_peered":"2024-09-12T07:20:52.505496+0000","last_clean":"2024-09-12T07:20:52.505496+0000","last_became_active":"2024-09-12T07:19:42.450577+0000","last_became_peered":"2024-09-12T07:19:42.450577+0000","last_unstale":"2024-09-12T07:20:52.505496+0000","last_undegraded":"2024-09-12T07:20:52.505496+0000","last_fullsized":"2024-09-12T07:20:52.505496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:26:36.110003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317836,"stat_sum":{"num_bytes":39092253,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":21,"num_read":1950,"num_read_kb":19995,"num_write":235,"num_write_kb":10818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"90'380","reported_seq":4101,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505529+0000","last_change":"2024-09-12T07:19:44.481850+0000","last_active":"2024-09-12T07:20:52.505529+0000","last_peered":"2024-09-12T07:20:52.505529+0000","last_clean":"2024-09-12T07:20:52.505529+0000","last_became_active":"2024-09-12T07:19:42.452236+0000","last_became_peered":"2024-09-12T07:19:42.452236+0000","last_unstale":"2024-09-12T07:20:52.505529+0000","last_undegraded":"2024-09-12T07:20:52.505529+0000","last_fullsized":"2024-09-12T07:20:52.505529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:29:18.645276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012939710000000001,"stat_sum":{"num_bytes":39322702,"num_objects":58,"num_object_clones":19,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":27,"num_read":5481,"num_read_kb":28298,"num_write":474,"num_write_kb":12384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"89'384","reported_seq":1513,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505487+0000","last_change":"2024-09-12T07:19:44.482179+0000","last_active":"2024-09-12T07:20:52.505487+0000","last_peered":"2024-09-12T07:20:52.505487+0000","last_clean":"2024-09-12T07:20:52.505487+0000","last_became_active":"2024-09-12T07:19:42.452155+0000","last_became_peered":"2024-09-12T07:19:42.452155+0000","last_unstale":"2024-09-12T07:20:52.505487+0000","last_undegraded":"2024-09-12T07:20:52.505487+0000","last_fullsized":"2024-09-12T07:20:52.505487+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:17.679427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011884759999999999,"stat_sum":{"num_bytes":29997006,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":18,"num_read":2292,"num_read_kb":11748,"num_write":469,"num_write_kb":15347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"91'221","reported_seq":1184,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.460050+0000","last_change":"2024-09-12T07:20:37.330497+0000","last_active":"2024-09-12T07:20:52.460050+0000","last_peered":"2024-09-12T07:20:52.460050+0000","last_clean":"2024-09-12T07:20:52.460050+0000","last_became_active":"2024-09-12T07:20:37.328982+0000","last_became_peered":"2024-09-12T07:20:37.328982+0000","last_unstale":"2024-09-12T07:20:52.460050+0000","last_undegraded":"2024-09-12T07:20:52.460050+0000","last_fullsized":"2024-09-12T07:20:52.460050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:53.950746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073032100000000003,"stat_sum":{"num_bytes":62792458,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":1170,"num_read_kb":23205,"num_write":243,"num_write_kb":16624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"90'218","reported_seq":3088,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505316+0000","last_change":"2024-09-12T07:19:44.481275+0000","last_active":"2024-09-12T07:20:52.505316+0000","last_peered":"2024-09-12T07:20:52.505316+0000","last_clean":"2024-09-12T07:20:52.505316+0000","last_became_active":"2024-09-12T07:19:42.452104+0000","last_became_peered":"2024-09-12T07:19:42.452104+0000","last_unstale":"2024-09-12T07:20:52.505316+0000","last_undegraded":"2024-09-12T07:20:52.505316+0000","last_fullsized":"2024-09-12T07:20:52.505316+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:23:47.134372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079083699999999996,"stat_sum":{"num_bytes":22840029,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":3219,"num_read_kb":34127,"num_write":203,"num_write_kb":16207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"90'207","reported_seq":2535,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.395478+0000","last_change":"2024-09-12T07:19:44.456086+0000","last_active":"2024-09-12T07:20:52.395478+0000","last_peered":"2024-09-12T07:20:52.395478+0000","last_clean":"2024-09-12T07:20:52.395478+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:20:52.395478+0000","last_undegraded":"2024-09-12T07:20:52.395478+0000","last_fullsized":"2024-09-12T07:20:52.395478+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:01.686363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060746,"stat_sum":{"num_bytes":33597399,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":3341,"num_read_kb":22721,"num_write":247,"num_write_kb":11637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"90'359","reported_seq":18790,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T07:20:52.505348+0000","last_change":"2024-09-12T07:19:44.482564+0000","last_active":"2024-09-12T07:20:52.505348+0000","last_peered":"2024-09-12T07:20:52.505348+0000","last_clean":"2024-09-12T07:20:52.505348+0000","last_became_active":"2024-09-12T07:19:42.456127+0000","last_became_peered":"2024-09-12T07:19:42.456127+0000","last_unstale":"2024-09-12T07:20:52.505348+0000","last_undegraded":"2024-09-12T07:20:52.505348+0000","last_fullsized":"2024-09-12T07:20:52.505348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:42:39.818004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793039999999999,"stat_sum":{"num_bytes":69124901,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":24,"num_read":18890,"num_read_kb":80661,"num_write":467,"num_write_kb":41147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1289771554,"num_objects":1514,"num_object_clones":367,"num_object_copies":3028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1514,"num_whiteouts":630,"num_read":190388,"num_read_kb":926033,"num_write":13898,"num_write_kb":493326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":64106948,"num_keys_recovered":75,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":653631488,"data_stored":793189781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11239,"ondisk_log_size":11239,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":151904,"kb_used_data":112388,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94219936,"statfs":{"total":96636764160,"available":96481214464,"internally_reserved":0,"allocated":115085312,"data_stored":138210280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542165,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55188,"kb_used_data":24004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316652,"statfs":{"total":96636764160,"available":96580251648,"internally_reserved":0,"allocated":24580096,"data_stored":23956063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":128132,"kb_used_data":88628,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94243708,"statfs":{"total":96636764160,"available":96505556992,"internally_reserved":0,"allocated":90755072,"data_stored":106737684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":191352,"kb_used_data":164320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94180488,"statfs":{"total":96636764160,"available":96440819712,"internally_reserved":0,"allocated":168263680,"data_stored":216390839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542165,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148272,"kb_used_data":99036,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94223568,"statfs":{"total":96636764160,"available":96484933632,"internally_reserved":0,"allocated":101412864,"data_stored":118182227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50104,"kb_used_data":14756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321736,"statfs":{"total":96636764160,"available":96585457664,"internally_reserved":0,"allocated":15110144,"data_stored":14698545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80220,"kb_used_data":53188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94291620,"statfs":{"total":96636764160,"available":96554618880,"internally_reserved":0,"allocated":54464512,"data_stored":68666701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542165,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117788,"kb_used_data":90756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94254052,"statfs":{"total":96636764160,"available":96516149248,"internally_reserved":0,"allocated":92934144,"data_stored":110804336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":91287552,"data_stored":109730779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53551104,"data_stored":68301979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14049280,"data_stored":14211392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":100560896,"data_stored":117876765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":167190528,"data_stored":215897844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":89915392,"data_stored":106459439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23572480,"data_stored":23519217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":113504256,"data_stored":137192366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:20:57.241 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-12T07:20:57.241 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:20:57.241 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-12T07:20:57.242 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-12T07:20:57.242 INFO:teuthology.orchestra.run.smithi046.stdout:5219 write 0x431eb92 thru 0x432a070 (0xb4df bytes) 2024-09-12T07:20:57.244 INFO:teuthology.orchestra.run.smithi046.stdout:5220 write 0xdb008f thru 0xdb218b (0x20fd bytes) 2024-09-12T07:20:57.245 INFO:teuthology.orchestra.run.smithi046.stdout:5221 punch from 0xbef71e9 to 0xbf0079b, (0x95b2 bytes) 2024-09-12T07:20:57.245 INFO:teuthology.orchestra.run.smithi046.stdout:5223 write 0xcb6b617 thru 0xcb6bb93 (0x57d bytes) 2024-09-12T07:20:57.246 INFO:teuthology.orchestra.run.smithi046.stdout:5224 write 0x3239a5 thru 0x32d809 (0x9e65 bytes) 2024-09-12T07:20:57.261 INFO:teuthology.orchestra.run.smithi046.stdout:5226 trunc from 0xd3f2e0d to 0x2ab8517 2024-09-12T07:20:57.283 INFO:teuthology.orchestra.run.smithi046.stdout:5229 writesame 0x55a2e61 thru 0x55a595a data_size 0x1(0x2afa bytes) 2024-09-12T07:20:57.286 INFO:teuthology.orchestra.run.smithi046.stdout:5230 trunc from 0x55a595b to 0x2c054ff 2024-09-12T07:20:57.296 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:57.297+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:57.310 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:57.310 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:57.312 INFO:teuthology.orchestra.run.smithi046.stdout:5234 write 0xe6c50f8 thru 0xe6d13f7 (0xc300 bytes) 2024-09-12T07:20:57.318 INFO:teuthology.orchestra.run.smithi046.stdout:5235 read 0x36d8320 thru 0x36e0fee (0x8ccf bytes) 2024-09-12T07:20:57.319 INFO:teuthology.orchestra.run.smithi046.stdout:5236 read 0x41f72ac thru 0x420349d (0xc1f2 bytes) 2024-09-12T07:20:57.319 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:57.319 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:57.319 INFO:teuthology.orchestra.run.smithi046.stdout:5239 read 0xa26e831 thru 0xa2782a5 (0x9a75 bytes) 2024-09-12T07:20:57.319 INFO:teuthology.orchestra.run.smithi046.stdout:5240 read 0x8311893 thru 0x8320874 (0xefe2 bytes) 2024-09-12T07:20:57.319 INFO:teuthology.orchestra.run.smithi046.stdout:5241 write 0x9bf4f34 thru 0x9c046eb (0xf7b8 bytes) 2024-09-12T07:20:57.321 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:20:57.335 INFO:teuthology.orchestra.run.smithi046.stdout:5242 trunc from 0xe6d13f8 to 0x43e1fa7 2024-09-12T07:20:57.348 INFO:teuthology.orchestra.run.smithi046.stdout:5243 trunc from 0x43e1fa7 to 0x1c67874 2024-09-12T07:20:57.355 INFO:teuthology.orchestra.run.smithi046.stdout:5244 clone 41 order 23 su 1048576 sc 11 2024-09-12T07:20:57.397 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:57.398+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:57.451 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:57.451 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:57.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:20:57.497 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:57.499+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:57.503 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone40 from 0x1c67874 (overlap 0x1f8d84) to 0xc9f01 2024-09-12T07:20:57.598 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-12T07:20:57.599+0000 7f0757424640 -1 mon.a@0(leader).osd e96 definitely_dead 0 2024-09-12T07:20:57.598 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:57.600+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:57.611 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:57.611 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:57.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:20:57.704 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:57.706+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:57.773 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:57.774 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:57.783 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:20:57.802 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-12T07:20:57.803 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:57.804+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:57.901 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:57.903+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:57.924 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:57.925 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:57.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:20:57.998 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:20:58.042 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:58.044+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:58.045 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:20:58.045 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:20:58.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:20:58.073 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:58.073 INFO:teuthology.orchestra.run.smithi046.stdout:5248 read 0xa48a0a thru 0xa4d77a (0x4d71 bytes) 2024-09-12T07:20:58.075 INFO:teuthology.orchestra.run.smithi046.stdout:5249 punch from 0x14554a5 to 0x145d622, (0x817d bytes) 2024-09-12T07:20:58.078 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:58.097 INFO:teuthology.orchestra.run.smithi046.stdout:5252 write 0xb8c4dc0 thru 0xb8caa57 (0x5c98 bytes) 2024-09-12T07:20:58.103 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:58.104+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:58.113 INFO:teuthology.orchestra.run.smithi046.stdout:5253 trunc from 0xb8caa58 to 0xf41854 2024-09-12T07:20:58.150 INFO:teuthology.orchestra.run.smithi046.stdout:5255 punch from 0x820496 to 0x822acc, (0x2636 bytes) 2024-09-12T07:20:58.157 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:20:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:20:58.160 INFO:teuthology.orchestra.run.smithi046.stdout:5256 write 0x22d04a2 thru 0x22d587a (0x53d9 bytes) 2024-09-12T07:20:58.168 INFO:teuthology.orchestra.run.smithi046.stdout:5258 write 0x23cda3b thru 0x23d6a8a (0x9050 bytes) 2024-09-12T07:20:58.170 INFO:teuthology.orchestra.run.smithi046.stdout:5259 read 0xf17da thru 0xf252b (0xd52 bytes) 2024-09-12T07:20:58.170 INFO:teuthology.orchestra.run.smithi046.stdout:5260 read 0xe67492 thru 0xe6f4bb (0x802a bytes) 2024-09-12T07:20:58.171 INFO:teuthology.orchestra.run.smithi046.stdout:5262 punch from 0x2181c23 to 0x2186356, (0x4733 bytes) 2024-09-12T07:20:58.179 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:20:58.180 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:20:58.192 INFO:teuthology.orchestra.run.smithi046.stdout:5263 write 0xdfba391 thru 0xdfc5d63 (0xb9d3 bytes) 2024-09-12T07:20:58.196 INFO:teuthology.orchestra.run.smithi046.stdout:5266 read 0x17c77a2 thru 0x17d50d8 (0xd937 bytes) 2024-09-12T07:20:58.196 INFO:teuthology.orchestra.run.smithi046.stdout:5267 trunc from 0xdfc5d64 to 0xda39d0b 2024-09-12T07:20:58.199 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:20:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:20:58.203 INFO:teuthology.orchestra.run.smithi046.stdout:5270 write 0x61c0c48 thru 0x61d0902 (0xfcbb bytes) 2024-09-12T07:20:58.203 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:58.205+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:58.207 INFO:teuthology.orchestra.run.smithi046.stdout:5274 punch from 0x271656e to 0x2719897, (0x3329 bytes) 2024-09-12T07:20:58.207 INFO:teuthology.orchestra.run.smithi046.stdout:5276 write 0x1edb9d2 thru 0x1edf85d (0x3e8c bytes) 2024-09-12T07:20:58.208 INFO:teuthology.orchestra.run.smithi046.stdout:5277 writesame 0xc88673b thru 0xc88904e data_size 0x16(0x2914 bytes) 2024-09-12T07:20:58.211 INFO:teuthology.orchestra.run.smithi046.stdout:5278 write 0x3bfe7e5 thru 0x3c00e25 (0x2641 bytes) 2024-09-12T07:20:58.213 INFO:teuthology.orchestra.run.smithi046.stdout:5279 read 0x7e462f thru 0x7edda4 (0x9776 bytes) 2024-09-12T07:20:58.214 INFO:teuthology.orchestra.run.smithi046.stdout:5280 write 0x937983e thru 0x9383f6a (0xa72d bytes) 2024-09-12T07:20:58.217 INFO:teuthology.orchestra.run.smithi046.stdout:5281 read 0x9c30fba thru 0x9c37352 (0x6399 bytes) 2024-09-12T07:20:58.217 INFO:teuthology.orchestra.run.smithi046.stdout:5282 read 0x90002c8 thru 0x9005df3 (0x5b2c bytes) 2024-09-12T07:20:58.217 INFO:teuthology.orchestra.run.smithi046.stdout:5283 punch from 0xb3dd0e7 to 0xb3e5f23, (0x8e3c bytes) 2024-09-12T07:20:58.218 INFO:teuthology.orchestra.run.smithi046.stdout:5285 read 0xd837435 thru 0xd84419c (0xcd68 bytes) 2024-09-12T07:20:58.218 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:58.218 INFO:teuthology.orchestra.run.smithi046.stdout:5289 read 0xc9fc12a thru 0xca00b5e (0x4a35 bytes) 2024-09-12T07:20:58.218 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:58.218 INFO:teuthology.orchestra.run.smithi046.stdout:5293 read 0xa065d8 thru 0xa13b5d (0xd586 bytes) 2024-09-12T07:20:58.218 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:20:58.218 INFO:teuthology.orchestra.run.smithi046.stdout:5295 write 0x600fc07 thru 0x6019935 (0x9d2f bytes) 2024-09-12T07:20:58.221 INFO:teuthology.orchestra.run.smithi046.stdout:5296 clone 42 order 21 su 1048576 sc 4 2024-09-12T07:20:58.222 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:20:58.222 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:20:58.224 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:20:58.224 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:20:58.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:20:58.224 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:20:58.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126196 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.703497959, 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038184672000000003, 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126196 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.225 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521349+0000", 2024-09-12T07:20:58.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6199999999999999e-07 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521351+0000", 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9520000000000001e-06 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521357+0000", 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521362+0000", 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9180000000000002e-06 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559327+0000", 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037964880999999999 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559401+0000", 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3750000000000004e-05 2024-09-12T07:20:58.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559533+0000", 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013262300000000001 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126334 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702917222, 2024-09-12T07:20:58.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038163516000000001, 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126334 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521929+0000", 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521930+0000", 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521932+0000", 2024-09-12T07:20:58.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9420000000000002e-06 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521938+0000", 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.127e-06 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521949+0000", 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1538e-05 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559814+0000", 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037864291000000001 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.559903+0000", 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9409999999999999e-05 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560093+0000", 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018984899999999999 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.233 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.234 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.234 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126336 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.234 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:58.234 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702888985000001, 2024-09-12T07:20:58.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038384952, 2024-09-12T07:20:58.234 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126336 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521958+0000", 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2e-07 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521962+0000", 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.591e-06 2024-09-12T07:20:58.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521969+0000", 2024-09-12T07:20:58.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9199999999999998e-06 2024-09-12T07:20:58.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521974+0000", 2024-09-12T07:20:58.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2179999999999998e-06 2024-09-12T07:20:58.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560137+0000", 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038163034999999998 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560193+0000", 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6425000000000003e-05 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560343+0000", 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000149343 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.238 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126337 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702865874, 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038587955, 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126337 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521981+0000", 2024-09-12T07:20:58.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:20:58.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521983+0000", 2024-09-12T07:20:58.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2079999999999999e-06 2024-09-12T07:20:58.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521989+0000", 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8649999999999998e-06 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.521994+0000", 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8720000000000001e-06 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560393+0000", 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038399232999999998 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560442+0000", 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8310999999999999e-05 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560569+0000", 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012708700000000001 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.243 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126339 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702846000000001, 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038809422000000003, 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126339 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522001+0000", 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.58e-07 2024-09-12T07:20:58.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522003+0000", 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0059999999999999e-06 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522009+0000", 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522014+0000", 2024-09-12T07:20:58.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0409999999999997e-06 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560605+0000", 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038590994000000003 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560650+0000", 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5336000000000001e-05 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560810+0000", 2024-09-12T07:20:58.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000159853 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126341 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702826671, 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039112779, 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.248 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126341 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522020+0000", 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6399999999999998e-07 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522022+0000", 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522028+0000", 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9089999999999997e-06 2024-09-12T07:20:58.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522035+0000", 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5200000000000003e-06 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560857+0000", 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038822239000000001 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.560955+0000", 2024-09-12T07:20:58.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7503999999999993e-05 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561133+0000", 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017827899999999999 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126342 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702804735000001, 2024-09-12T07:20:58.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039296258000000001, 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126342 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522042+0000", 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.84e-07 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522044+0000", 2024-09-12T07:20:58.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-12T07:20:58.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522057+0000", 2024-09-12T07:20:58.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330999999999999e-05 2024-09-12T07:20:58.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522062+0000", 2024-09-12T07:20:58.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3700000000000003e-06 2024-09-12T07:20:58.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561151+0000", 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039089193000000001 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561215+0000", 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3305000000000005e-05 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561338+0000", 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012353900000000001 2024-09-12T07:20:58.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126343 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702776617000001, 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039573481000000001, 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126343 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522070+0000", 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522071+0000", 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2599999999999998e-07 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522077+0000", 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2049999999999999e-06 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522084+0000", 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1679999999999998e-06 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522089+0000", 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561375+0000", 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039286692999999998 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561463+0000", 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7575000000000001e-05 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561644+0000", 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018070799999999999 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126344 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702750100999999, 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039759638, 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126344 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522097+0000", 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4900000000000001e-07 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522099+0000", 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4250000000000001e-06 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522106+0000", 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.455e-06 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522112+0000", 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3169999999999998e-06 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561676+0000", 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039564153999999997 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561728+0000", 2024-09-12T07:20:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1848999999999998e-05 2024-09-12T07:20:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561856+0000", 2024-09-12T07:20:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012808900000000001 2024-09-12T07:20:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126345 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702723507999998, 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040055773000000003, 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126345 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522123+0000", 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522124+0000", 2024-09-12T07:20:58.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6600000000000002e-07 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522130+0000", 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.72e-06 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522140+0000", 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0217000000000001e-05 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522146+0000", 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3380000000000004e-06 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561896+0000", 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039750343 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.561997+0000", 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010129899999999999 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562179+0000", 2024-09-12T07:20:58.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018149 2024-09-12T07:20:58.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.270 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.270 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126346 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.270 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:58.270 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702516273000001, 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040058279000000002, 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126346 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522330+0000", 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522331+0000", 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522334+0000", 2024-09-12T07:20:58.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.01e-06 2024-09-12T07:20:58.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522346+0000", 2024-09-12T07:20:58.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:20:58.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522353+0000", 2024-09-12T07:20:58.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5590000000000002e-06 2024-09-12T07:20:58.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562212+0000", 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039859232000000001 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562256+0000", 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4354999999999998e-05 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562389+0000", 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013237899999999999 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126348 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702483588, 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040352420999999999, 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.275 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.276 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126348 2024-09-12T07:20:58.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.276 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522363+0000", 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522364+0000", 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.51e-07 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522366+0000", 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522373+0000", 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3339999999999996e-06 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522378+0000", 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8620000000000002e-06 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562428+0000", 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040049840000000003 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562522+0000", 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3589000000000001e-05 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562716+0000", 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019395799999999999 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.279 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126349 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1269760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702460779999999, 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040535117000000002, 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126349 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522386+0000", 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522389+0000", 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.317e-06 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522396+0000", 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:58.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522401+0000", 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5770000000000001e-06 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562754+0000", 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040352271000000002 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562809+0000", 2024-09-12T07:20:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5376e-05 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562921+0000", 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112098 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126350 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:58.284 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702438291, 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040836117999999998, 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126350 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:58.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522408+0000", 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522409+0000", 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9099999999999999e-07 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522411+0000", 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1449999999999998e-06 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522417+0000", 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1070000000000001e-06 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522422+0000", 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7600000000000002e-06 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.562965+0000", 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040543467 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563058+0000", 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2905000000000006e-05 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563245+0000", 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186343 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.288 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126351 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702247450000002, 2024-09-12T07:20:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040915973000000001, 2024-09-12T07:20:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.289 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126351 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522599+0000", 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522600+0000", 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7899999999999998e-07 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522604+0000", 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-06 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522615+0000", 2024-09-12T07:20:58.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1754e-05 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522622+0000", 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.798e-06 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563277+0000", 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040654765000000002 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563340+0000", 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2580000000000001e-05 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563515+0000", 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017560699999999999 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.293 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126352 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.294 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:58.294 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702212156000002, 2024-09-12T07:20:58.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041184365000000001, 2024-09-12T07:20:58.294 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.294 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.294 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.294 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.294 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.294 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126352 2024-09-12T07:20:58.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.294 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522635+0000", 2024-09-12T07:20:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.99e-07 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522637+0000", 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3180000000000001e-06 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522645+0000", 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0309999999999998e-06 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522651+0000", 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.789e-06 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563559+0000", 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.040907261 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563634+0000", 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5708000000000001e-05 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563819+0000", 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000184659 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126353 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702186702999999, 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388632000000002, 2024-09-12T07:20:58.299 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126353 2024-09-12T07:20:58.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.300 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:58.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522660+0000", 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522661+0000", 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7299999999999996e-07 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522663+0000", 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5979999999999999e-06 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522670+0000", 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6850000000000004e-06 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522675+0000", 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9139999999999999e-06 2024-09-12T07:20:58.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563854+0000", 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041178851000000002 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.563912+0000", 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8584e-05 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564049+0000", 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000136427 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:126354 3.18 3:1e002a1d:::rbd_data.116067a48597.0000000000000000:19 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.702164177, 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041618080000000002, 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 126354 2024-09-12T07:20:58.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.305 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:58.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522683+0000", 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2399999999999999e-07 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522685+0000", 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2469999999999998e-06 2024-09-12T07:20:58.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522691+0000", 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.353e-06 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.522697+0000", 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.113e-06 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564085+0000", 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041388104000000002 2024-09-12T07:20:58.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564149+0000", 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4471e-05 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.564301+0000", 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015146800000000001 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.308 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:129861 3.18 3:1cf16ce6:::rbd_data.116031710b3d.00000000000000ae:18 [read 40960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.452228466000001, 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038044815000000003, 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 129861 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772618+0000", 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772619+0000", 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5400000000000002e-07 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772620+0000", 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6980000000000001e-06 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772627+0000", 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3119999999999999e-06 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772631+0000", 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9010000000000002e-06 2024-09-12T07:20:58.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810471+0000", 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037839308000000002 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810521+0000", 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9988999999999999e-05 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810663+0000", 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014225300000000001 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:129880 3.8 3:149a3dec:::rbd_data.116031710b3d.00000000000000ab:18 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.452188249999999, 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038109654999999999, 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:20:58.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 129880 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772658+0000", 2024-09-12T07:20:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772659+0000", 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772661+0000", 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8279999999999999e-06 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:20:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772666+0000", 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6319999999999996e-06 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.772672+0000", 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0510000000000002e-06 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810577+0000", 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.037904895000000001 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810688+0000", 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011072400000000001 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:27.810768+0000", 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0139e-05 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:20:58.317 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:20:58.319 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:20:58.320 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:20:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:20:58.320 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:20:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:20:58.321 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:20:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:20:58.321 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:20:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:20:58.322 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:58.321+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:20:58.420 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:58.421+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:58.429 INFO:teuthology.orchestra.run.smithi046.stderr:marked out osd.1. 2024-09-12T07:20:58.431 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:58.431 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:58.440 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:20:58.442 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:20:58.521 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:58.522+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:58.549 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:58.549 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:58.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:20:58.622 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:58.623+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:58.665 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 61.053169150000002, 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:20:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.917708894, 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:20:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:20:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.913481773000001, 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:20:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:20:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.185731118, 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:20:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.096754529000002, 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68853 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'43, mlcod=43'39)", 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.085761941999998, 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017793166999999999, 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580718+0000", 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580719+0000", 2024-09-12T07:20:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4720000000000001e-06 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580732+0000", 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2926e-05 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580746+0000", 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3291e-05 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.580755+0000", 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6709999999999994e-06 2024-09-12T07:20:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583766+0000", 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003010432 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.583794+0000", 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.811e-05 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598408+0000", 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014614472 2024-09-12T07:20:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.598511+0000", 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102793 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.065203199000003, 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:20:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.062002663000001, 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:20:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:20:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:20:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:20:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.058756914999996, 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:20:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:20:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:20:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:20:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:20:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:20:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:20:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.056515083999997, 2024-09-12T07:20:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:20:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:20:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:20:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69559 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'29, mlcod=41'27)", 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 54.982184036, 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018119258999999999, 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684296+0000", 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.684297+0000", 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.192e-06 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691509+0000", 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0072122239999999997 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691524+0000", 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4562999999999999e-05 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691533+0000", 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1069999999999994e-06 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691598+0000", 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4666999999999996e-05 2024-09-12T07:20:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691620+0000", 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2092000000000001e-05 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702352+0000", 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010731785000000001 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702415+0000", 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3628999999999997e-05 2024-09-12T07:20:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69560 3.1a e43/24 3:5c26efc3:::rbd_data.1160f022dd61.0000000000000012:head v 43'31, mlcod=41'27)", 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 54.974617913000003, 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016047555000000002, 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691862+0000", 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691863+0000", 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0299999999999999e-07 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691897+0000", 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4765999999999999e-05 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691926+0000", 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8413000000000001e-05 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.691934+0000", 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3839999999999997e-06 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694739+0000", 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0028043640000000002 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.694773+0000", 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4514000000000002e-05 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707854+0000", 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013080591000000001 2024-09-12T07:20:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707910+0000", 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.592e-05 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "pg_update_log_missing(3.1a epoch 43/24 rep_tid 362 entries 43'32 (0'0) error 3:5ec4a676:::rbd_data.1160f022dd61.0000000000000019:head by client.4448.0:69639 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 41'27)", 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 54.974090199000003, 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.015537618, 2024-09-12T07:20:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692390+0000", 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9999999999999997e-07 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692497+0000", 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107154 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692519+0000", 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1043000000000002e-05 2024-09-12T07:20:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.692542+0000", 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3866000000000002e-05 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.702496+0000", 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0099539169999999996 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.707927+0000", 2024-09-12T07:20:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005431038 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 54.965407880000001, 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:20:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:20:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:20:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:20:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:20:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:20:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:20:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 54.961579792000002, 2024-09-12T07:20:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:20:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:20:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89144 3.a e53/24 3:54996542:::rbd_data.11607a56846.0000000000000001:head v 53'83, mlcod=53'81)", 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 44.973423038, 2024-09-12T07:20:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018298937000000001, 2024-09-12T07:20:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693057+0000", 2024-09-12T07:20:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.693059+0000", 2024-09-12T07:20:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.565e-06 2024-09-12T07:20:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696238+0000", 2024-09-12T07:20:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0031783269999999999 2024-09-12T07:20:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696265+0000", 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6982e-05 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696279+0000", 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4438e-05 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696366+0000", 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6978000000000006e-05 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.696397+0000", 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0406999999999999e-05 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711297+0000", 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014900689999999999 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.711356+0000", 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.855e-05 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89145 3.13 e53/24 3:ca7c3842:::rbd_data.11607a56846.0000000000000002:head v 53'202, mlcod=53'199)", 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 44.972391694999999, 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019835603, 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694091+0000", 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4990000000000001e-06 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697616+0000", 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035256749999999998 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697635+0000", 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8593999999999998e-05 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697646+0000", 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1436e-05 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697717+0000", 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0679000000000005e-05 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697748+0000", 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0723999999999999e-05 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713886+0000", 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016138550000000002 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713924+0000", 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7446e-05 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 31.979667011, 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:20:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 31.967853367, 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:20:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 31.966517320000001, 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:20:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:20:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:58.753 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:58.754 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:20:58.755 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:20:58.756+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:20:58.854 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:58.856+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:58.902 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:58.903 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:58.911 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:20:58.995 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:20:58.997+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:20:59.056 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:20:59.057+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:20:59.058 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:59.059 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:59.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:20:59.156 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:59.158+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:59.215 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.734774184000003, 2024-09-12T07:20:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:20:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:20:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.734747304000003, 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.734723283999998, 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:20:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:20:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.734695637999998, 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:20:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.734672175999997, 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:20:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:20:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.734649648999998, 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:20:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:20:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.734627846000002, 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:20:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.727994066000001, 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.727974058000001, 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:20:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:20:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:20:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:20:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:20:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:20:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:20:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:20:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.721491026000002, 2024-09-12T07:20:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.721455739, 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:20:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.721433105999999, 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:20:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:20:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:20:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:20:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:20:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:20:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.721379051, 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:20:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:20:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:20:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:20:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:20:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:20:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:20:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:20:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:20:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:20:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.721360109999999, 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:20:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.721348173000003, 2024-09-12T07:20:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:20:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:20:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:20:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.721336557999997, 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:20:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:20:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:20:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:20:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:20:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.714323950000001, 2024-09-12T07:20:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:20:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:20:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:20:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:20:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:20:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:20:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:20:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:20:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:20:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:20:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.714300235000003, 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:20:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:20:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:20:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 59.714169378000001, 2024-09-12T07:20:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:20:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 55.750498587999999, 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:20:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:20:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:20:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:20:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:20:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:20:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.309 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:59.311 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:20:59.311 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:59.313+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:59.411 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:59.413+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:59.427 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:20:59.462 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:59.462 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:59.462 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:20:59.462 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:59.471 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:20:59.512 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:20:59.514+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:20:59.614 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:20:59.616+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:20:59.618 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:20:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:20:59.618 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:20:59.619 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:59.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:20:59.715 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:20:59.717+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:20:59.823 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:20:59.823 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:20:59.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:20:59.823 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:20:59.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.823 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:20:59.823 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:59.823 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 67.207648586999994, 2024-09-12T07:20:59.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:59.828 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.355850920999998, 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:59.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:20:59.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.832 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.355727643999998, 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.833 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:20:59.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.355524252999999, 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.841 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.355388425999998, 2024-09-12T07:20:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:20:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:20:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:20:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:20:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:20:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.846 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.355061769999999, 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:59.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:20:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:20:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:20:59.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:59.851 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.355022882, 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:59.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:20:59.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.354656007999999, 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.856 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:20:59.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:20:59.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:20:59.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.354521151999997, 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:20:59.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:20:59.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.354151260999998, 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:20:59.864 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:20:59.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:20:59.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.353961063, 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:20:59.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:20:59.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:20:59.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.353740983999998, 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.873 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:20:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.353597319000002, 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:20:59.877 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.881 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.353371146999997, 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.882 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:20:59.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:20:59.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.353170789000004, 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.886 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:20:59.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:20:59.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.352762906000002, 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:20:59.890 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:20:59.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:20:59.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.352297024000002, 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.895 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.897 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:20:59.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.350694814000001, 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:20:59.899 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.350160778999999, 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:20:59.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.093962709000003, 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:20:59.908 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:20:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:20:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:20:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:20:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:20:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:20:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:20:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:20:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:20:59.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.912 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:20:59.913 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:20:59.913 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:20:59.913 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:20:59.914 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:20:59.824+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:20:59.956 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:20:59.958+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:00.020 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:00.020 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:00.021 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:00.021 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:00.023 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:00.024+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:00.029 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:00.124 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:00.125+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:00.175 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:00.175 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:00.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:00.225 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:00.226+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:00.292 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.292 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743946020000003, 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:21:00.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743925482999998, 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.297 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:21:00.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743905480000002, 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.302 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:21:00.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743885530999997, 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:21:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.310 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743864662999997, 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:21:00.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:21:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:21:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:21:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:21:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:21:00.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743842962000002, 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.315 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.319 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743822522000002, 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:21:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743794801, 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.324 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:21:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743771866000003, 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:21:00.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743751566, 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:21:00.333 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:21:00.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:21:00.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743731289999999, 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.338 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:21:00.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:00.342 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743711130999998, 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743573933, 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:21:00.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:21:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:21:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:21:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:21:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:21:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.351 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.743551285000002, 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:21:00.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:21:00.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68088 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 393216~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.737108526, 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10346986399999999, 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.356 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68088 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555923+0000", 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.891e-06 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555929+0000", 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0549999999999996e-06 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555934+0000", 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6419999999999998e-06 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659060+0000", 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103125862 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659139+0000", 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9450000000000007e-05 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659391+0000", 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025158400000000001 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.360 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.737089298999997, 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:21:00.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:21:00.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.736859662000001, 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.365 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:00.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.369 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.729935971000003, 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:21:00.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.729576387999998, 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.378 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 56.729496402000002, 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:00.379 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:21:00.382 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:00.383 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:00.383 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:00.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:00.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:00.386 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:00.386+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:00.485 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:00.487+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:00.520 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:00.521 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:00.521 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:00.531 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:00.538 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:00.557 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:00.576 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:00.577 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:00.586 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:00.587+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:00.595 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:00.595 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:00.614 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:00.633 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:00.651 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:00.655 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:00.655 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:00.665 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:00.671 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:00.671 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:00.725 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:00.727+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:00.787 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:00.788+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:00.797 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:00.797 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:00.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:00.885 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone41 from 0xda39d0b (overlap 0xf41854) to 0x1868d5 2024-09-12T07:21:00.888 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:00.889+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:00.989 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:00.991+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:00.994 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:00.995 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:01.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:01.027 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-12T07:21:01.033 INFO:teuthology.orchestra.run.smithi046.stdout:5297 read 0xd3b23a9 thru 0xd3bd92b (0xb583 bytes) 2024-09-12T07:21:01.034 INFO:teuthology.orchestra.run.smithi046.stdout:5298 writesame 0x1e035ec thru 0x1e08499 data_size 0x1b(0x4eae bytes) 2024-09-12T07:21:01.043 INFO:teuthology.orchestra.run.smithi046.stdout:5299 writesame 0x214b7c3 thru 0x215229b data_size 0x1(0x6ad9 bytes) 2024-09-12T07:21:01.061 INFO:teuthology.orchestra.run.smithi046.stdout:5300 read 0x9863195 thru 0x9867374 (0x41e0 bytes) 2024-09-12T07:21:01.061 INFO:teuthology.orchestra.run.smithi046.stdout:5301 read 0x63e02f9 thru 0x63e321f (0x2f27 bytes) 2024-09-12T07:21:01.061 INFO:teuthology.orchestra.run.smithi046.stdout:5303 write 0x6a8fae5 thru 0x6a9096a (0xe86 bytes) 2024-09-12T07:21:01.065 INFO:teuthology.orchestra.run.smithi046.stdout:5305 read 0x4c5d2a thru 0x4c8f67 (0x323e bytes) 2024-09-12T07:21:01.065 INFO:teuthology.orchestra.run.smithi046.stdout:5306 write 0xfeaf5b thru 0xff377d (0x8823 bytes) 2024-09-12T07:21:01.069 INFO:teuthology.orchestra.run.smithi046.stdout:5308 read 0xbd7ac12 thru 0xbd8a099 (0xf488 bytes) 2024-09-12T07:21:01.069 INFO:teuthology.orchestra.run.smithi046.stdout:5309 punch from 0x49217b1 to 0x4927879, (0x60c8 bytes) 2024-09-12T07:21:01.069 INFO:teuthology.orchestra.run.smithi046.stdout:5312 writesame 0x6693b9d thru 0x66a36a8 data_size 0x2(0xfb0c bytes) 2024-09-12T07:21:01.090 INFO:teuthology.orchestra.run.smithi046.stdout:5313 read 0xd31ba91 thru 0xd3268f8 (0xae68 bytes) 2024-09-12T07:21:01.090 INFO:teuthology.orchestra.run.smithi046.stdout:5320 punch from 0xc1bcd73 to 0xc1bde64, (0x10f1 bytes) 2024-09-12T07:21:01.090 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:01.090 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:01.090 INFO:teuthology.orchestra.run.smithi046.stdout:5323 punch from 0x7967ae3 to 0x796d30a, (0x5827 bytes) 2024-09-12T07:21:01.090 INFO:teuthology.orchestra.run.smithi046.stdout:5324 flatten 2024-09-12T07:21:01.091 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:01.091+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:01.119 INFO:teuthology.orchestra.run.smithi046.stdout:5325 punch from 0x7b7b6cf to 0x7b801ae, (0x4adf bytes) 2024-09-12T07:21:01.119 INFO:teuthology.orchestra.run.smithi046.stdout:5326 writesame 0x42334c4 thru 0x423954b data_size 0x4(0x6088 bytes) 2024-09-12T07:21:01.138 INFO:teuthology.orchestra.run.smithi046.stdout:5327 trunc from 0xda39d0b to 0x2930dd3 2024-09-12T07:21:01.159 INFO:teuthology.orchestra.run.smithi046.stdout:5328 trunc from 0x2930dd3 to 0xa45ae24 2024-09-12T07:21:01.162 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:01.162 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:01.172 INFO:teuthology.orchestra.run.smithi046.stdout:5330 trunc from 0xa45ae24 to 0x3bf5ce6 2024-09-12T07:21:01.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:01.185 INFO:teuthology.orchestra.run.smithi046.stdout:5333 trunc from 0x3bf5ce6 to 0xa2fd0a4 2024-09-12T07:21:01.190 INFO:teuthology.orchestra.run.smithi046.stdout:5334 writesame 0x2b901c8 thru 0x2b9a5b0 data_size 0x47(0xa3e9 bytes) 2024-09-12T07:21:01.190 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:01.192+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:01.196 INFO:teuthology.orchestra.run.smithi046.stdout:5335 trunc from 0xa2fd0a4 to 0xceb2d21 2024-09-12T07:21:01.202 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:01.202 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:01.202 INFO:teuthology.orchestra.run.smithi046.stdout:5338 write 0x14d1733 thru 0x14dc774 (0xb042 bytes) 2024-09-12T07:21:01.205 INFO:teuthology.orchestra.run.smithi046.stdout:5339 writesame 0x249fee5 thru 0x24ab161 data_size 0x3(0xb27d bytes) 2024-09-12T07:21:01.216 INFO:teuthology.orchestra.run.smithi046.stdout:5340 write 0x1e9dc01 thru 0x1eaa6f3 (0xcaf3 bytes) 2024-09-12T07:21:01.217 INFO:teuthology.orchestra.run.smithi046.stdout:5341 write 0x12e619d thru 0x12efa4f (0x98b3 bytes) 2024-09-12T07:21:01.228 INFO:teuthology.orchestra.run.smithi046.stdout:5343 writesame 0xe892b9e thru 0xe895495 data_size 0x4c(0x28f8 bytes) 2024-09-12T07:21:01.228 INFO:teuthology.orchestra.run.smithi046.stdout:5345 read 0xb8b5d33 thru 0xb8bf918 (0x9be6 bytes) 2024-09-12T07:21:01.228 INFO:teuthology.orchestra.run.smithi046.stdout:5346 read 0x6249b50 thru 0x624d52d (0x39de bytes) 2024-09-12T07:21:01.228 INFO:teuthology.orchestra.run.smithi046.stdout:5347 read 0xa62daf thru 0xa71f0f (0xf161 bytes) 2024-09-12T07:21:01.241 INFO:teuthology.orchestra.run.smithi046.stdout:5348 trunc from 0xe895496 to 0x2f52615 2024-09-12T07:21:01.247 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:01.247 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:01.251 INFO:teuthology.orchestra.run.smithi046.stdout:5351 trunc from 0x2f52615 to 0x6c37a5f 2024-09-12T07:21:01.259 INFO:teuthology.orchestra.run.smithi046.stdout:5352 trunc from 0x6c37a5f to 0x4febc50 2024-09-12T07:21:01.281 INFO:teuthology.orchestra.run.smithi046.stdout:5353 writesame 0xeaf27eb thru 0xeaf71ba data_size 0x8(0x49d0 bytes) 2024-09-12T07:21:01.281 INFO:teuthology.orchestra.run.smithi046.stdout:5354 write 0x324d1e0 thru 0x3254dea (0x7c0b bytes) 2024-09-12T07:21:01.284 INFO:teuthology.orchestra.run.smithi046.stdout:5355 write 0xa7dceb2 thru 0xa7eb4f0 (0xe63f bytes) 2024-09-12T07:21:01.287 INFO:teuthology.orchestra.run.smithi046.stdout:5356 writesame 0x8c0d0a1 thru 0x8c11931 data_size 0x1(0x4891 bytes) 2024-09-12T07:21:01.291 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:01.293+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:01.298 INFO:teuthology.orchestra.run.smithi046.stdout:5357 write 0x313e999 thru 0x3141284 (0x28ec bytes) 2024-09-12T07:21:01.301 INFO:teuthology.orchestra.run.smithi046.stdout:5358 punch from 0xa56d5e5 to 0xa5745d9, (0x6ff4 bytes) 2024-09-12T07:21:01.302 INFO:teuthology.orchestra.run.smithi046.stdout:5359 read 0x7978b6b thru 0x7985e3d (0xd2d3 bytes) 2024-09-12T07:21:01.302 INFO:teuthology.orchestra.run.smithi046.stdout:5361 read 0x9820859 thru 0x982695b (0x6103 bytes) 2024-09-12T07:21:01.302 INFO:teuthology.orchestra.run.smithi046.stdout:5362 read 0x3ca38ca thru 0x3ca3e87 (0x5be bytes) 2024-09-12T07:21:01.306 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:01.307 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:01.311 INFO:teuthology.orchestra.run.smithi046.stdout:5365 trunc from 0xeaf71bb to 0x5a1032a 2024-09-12T07:21:01.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:21:01.322 INFO:teuthology.orchestra.run.smithi046.stdout:5366 punch from 0x2a42a17 to 0x2a4d2e6, (0xa8cf bytes) 2024-09-12T07:21:01.322 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:01.337 INFO:teuthology.orchestra.run.smithi046.stdout:5368 writesame 0xd927981 thru 0xd92cca9 data_size 0x1(0x5329 bytes) 2024-09-12T07:21:01.337 INFO:teuthology.orchestra.run.smithi046.stdout:5369 read 0x96f110a thru 0x96f1a23 (0x91a bytes) 2024-09-12T07:21:01.342 INFO:teuthology.orchestra.run.smithi046.stdout:5370 trunc from 0xd92ccaa to 0x8fad4aa 2024-09-12T07:21:01.347 INFO:teuthology.orchestra.run.smithi046.stdout:5371 read 0x77a2eb4 thru 0x77b0921 (0xda6e bytes) 2024-09-12T07:21:01.347 INFO:teuthology.orchestra.run.smithi046.stdout:5374 read 0x69ffa1e thru 0x6a0ea25 (0xf008 bytes) 2024-09-12T07:21:01.356 INFO:teuthology.orchestra.run.smithi046.stdout:5375 write 0xc7b734c thru 0xc7b8d7a (0x1a2f bytes) 2024-09-12T07:21:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:5376 read 0x25f02ed thru 0x25fc4b4 (0xc1c8 bytes) 2024-09-12T07:21:01.360 INFO:teuthology.orchestra.run.smithi046.stdout:5378 read 0x515aafe thru 0x51618bd (0x6dc0 bytes) 2024-09-12T07:21:01.371 INFO:teuthology.orchestra.run.smithi046.stdout:5379 trunc from 0xc7b8d7b to 0x27a23f5 2024-09-12T07:21:01.392 INFO:teuthology.orchestra.run.smithi046.stdout:5380 writesame 0x655ec06 thru 0x655f5b2 data_size 0x1(0x9ad bytes) 2024-09-12T07:21:01.392 INFO:teuthology.orchestra.run.smithi046.stdout:5381 write 0x1b8c6d1 thru 0x1b97605 (0xaf35 bytes) 2024-09-12T07:21:01.392 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:01.394+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:01.395 INFO:teuthology.orchestra.run.smithi046.stdout:5382 read 0x377241a thru 0x377cc54 (0xa83b bytes) 2024-09-12T07:21:01.395 INFO:teuthology.orchestra.run.smithi046.stdout:5383 read 0xd35c03 thru 0xd42c2e (0xd02c bytes) 2024-09-12T07:21:01.396 INFO:teuthology.orchestra.run.smithi046.stdout:5385 write 0xb31c2e thru 0xb3c153 (0xa526 bytes) 2024-09-12T07:21:01.412 INFO:teuthology.orchestra.run.smithi046.stdout:5386 write 0xdd9e9e3 thru 0xddae8f9 (0xff17 bytes) 2024-09-12T07:21:01.415 INFO:teuthology.orchestra.run.smithi046.stdout:5388 read 0x318cfd4 thru 0x319b303 (0xe330 bytes) 2024-09-12T07:21:01.428 INFO:teuthology.orchestra.run.smithi046.stdout:5389 trunc from 0xddae8fa to 0x299a390 2024-09-12T07:21:01.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:21:01.453 INFO:teuthology.orchestra.run.smithi046.stdout:5391 write 0x9626d2d thru 0x9635ee5 (0xf1b9 bytes) 2024-09-12T07:21:01.453 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:01.453 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:01.460 INFO:teuthology.orchestra.run.smithi046.stdout:5392 write 0x23f4c06 thru 0x2400f76 (0xc371 bytes) 2024-09-12T07:21:01.463 INFO:teuthology.orchestra.run.smithi046.stdout:5393 write 0x23446e8 thru 0x23492c0 (0x4bd9 bytes) 2024-09-12T07:21:01.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:21:01.469 INFO:teuthology.orchestra.run.smithi046.stdout:5394 trunc from 0x9635ee6 to 0x54c1663 2024-09-12T07:21:01.493 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:01.495+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:01.584 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:01.585 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:01.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:21:01.594 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:01.597+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:01.695 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:01.697+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:01.701 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:01.701 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:01.710 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:01.796 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:01.798+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:01.817 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:01.817 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:01.826 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:01.897 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:01.898+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:01.933 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:01.933 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:01.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:01.998 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:02.000+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:02.090 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:02.090 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:02.099 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:02.101+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:02.100 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:02.199 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:02.201+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:02.218 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:02.219 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:02.227 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:02.301 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:02.302+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:02.376 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:02.376 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:02.384 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:02.402 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:02.403+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:21:02.474 INFO:teuthology.orchestra.run.smithi046.stdout:5395 trunc from 0x54c1663 to 0xd327163 2024-09-12T07:21:02.480 INFO:teuthology.orchestra.run.smithi046.stdout:5396 write 0x69cfce2 thru 0x69ddea9 (0xe1c8 bytes) 2024-09-12T07:21:02.483 INFO:teuthology.orchestra.run.smithi046.stdout:5397 read 0x1b8abf7 thru 0x1b950aa (0xa4b4 bytes) 2024-09-12T07:21:02.487 INFO:teuthology.orchestra.run.smithi046.stdout:5398 trunc from 0xd327163 to 0xa4ce9a7 2024-09-12T07:21:02.492 INFO:teuthology.orchestra.run.smithi046.stdout:5399 write 0x6af7883 thru 0x6b02162 (0xa8e0 bytes) 2024-09-12T07:21:02.495 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:02.495 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:02.496 INFO:teuthology.orchestra.run.smithi046.stdout:5400 write 0x2d7a10d thru 0x2d85253 (0xb147 bytes) 2024-09-12T07:21:02.503 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:02.504+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:02.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:02.510 INFO:teuthology.orchestra.run.smithi046.stdout:5401 writesame 0xe8b6d88 thru 0xe8c4fc5 data_size 0x2a(0xe23e bytes) 2024-09-12T07:21:02.525 INFO:teuthology.orchestra.run.smithi046.stdout:5404 trunc from 0xe8c4fc6 to 0x25bfd6d 2024-09-12T07:21:02.537 INFO:teuthology.orchestra.run.smithi046.stdout:5405 trunc from 0x25bfd6d to 0xc504a4 2024-09-12T07:21:02.554 INFO:teuthology.orchestra.run.smithi046.stdout:5406 trunc from 0xc504a4 to 0x3efcb96 2024-09-12T07:21:02.563 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:02.563 INFO:teuthology.orchestra.run.smithi046.stdout:5408 read 0x13a9af0 thru 0x13b0ae4 (0x6ff5 bytes) 2024-09-12T07:21:02.563 INFO:teuthology.orchestra.run.smithi046.stdout:5412 punch from 0x1b1b883 to 0x1b1be2c, (0x5a9 bytes) 2024-09-12T07:21:02.563 INFO:teuthology.orchestra.run.smithi046.stdout:5413 writesame 0x16da0df thru 0x16dc680 data_size 0x1(0x25a2 bytes) 2024-09-12T07:21:02.582 INFO:teuthology.orchestra.run.smithi046.stdout:5414 write 0x96b9f7d thru 0x96c2bc7 (0x8c4b bytes) 2024-09-12T07:21:02.585 INFO:teuthology.orchestra.run.smithi046.stdout:5416 read 0x4236ad2 thru 0x42394ef (0x2a1e bytes) 2024-09-12T07:21:02.585 INFO:teuthology.orchestra.run.smithi046.stdout:5417 punch from 0x59e4744 to 0x59ec58a, (0x7e46 bytes) 2024-09-12T07:21:02.592 INFO:teuthology.orchestra.run.smithi046.stdout:5418 trunc from 0x96c2bc8 to 0x31ae4d8 2024-09-12T07:21:02.603 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:02.605+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:02.611 INFO:teuthology.orchestra.run.smithi046.stdout:5420 writesame 0x6fd37d2 thru 0x6fe0bae data_size 0x65(0xd3dd bytes) 2024-09-12T07:21:02.611 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:02.611 INFO:teuthology.orchestra.run.smithi046.stdout:5422 writesame 0xe62e5e thru 0xe63b34 data_size 0x1(0xcd7 bytes) 2024-09-12T07:21:02.615 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:02.615 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:02.623 INFO:teuthology.orchestra.run.smithi046.stdout:5423 write 0xa4a5695 thru 0xa4b3e17 (0xe783 bytes) 2024-09-12T07:21:02.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:02.626 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:02.626 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:02.626 INFO:teuthology.orchestra.run.smithi046.stdout:5427 read 0x7f40292 thru 0x7f4b13e (0xaead bytes) 2024-09-12T07:21:02.632 INFO:teuthology.orchestra.run.smithi046.stdout:5429 trunc from 0xa4b3e18 to 0x4ad46e6 2024-09-12T07:21:02.657 INFO:teuthology.orchestra.run.smithi046.stdout:5432 write 0xd837e1d thru 0xd83cfa9 (0x518d bytes) 2024-09-12T07:21:02.660 INFO:teuthology.orchestra.run.smithi046.stdout:5433 write 0x78e6619 thru 0x78f37fd (0xd1e5 bytes) 2024-09-12T07:21:02.663 INFO:teuthology.orchestra.run.smithi046.stdout:5435 punch from 0x37a7378 to 0x37af5d5, (0x825d bytes) 2024-09-12T07:21:02.671 INFO:teuthology.orchestra.run.smithi046.stdout:5436 trunc from 0xd83cfaa to 0x6a3f642 2024-09-12T07:21:02.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:02.690 INFO:teuthology.orchestra.run.smithi046.stdout:5437 write 0xab13d94 thru 0xab2160a (0xd877 bytes) 2024-09-12T07:21:02.700 INFO:teuthology.orchestra.run.smithi046.stdout:5438 trunc from 0xab2160b to 0x541ec7f 2024-09-12T07:21:02.704 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:02.705+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:02.709 INFO:teuthology.orchestra.run.smithi046.stdout:5439 write 0x2096f7a thru 0x2099faa (0x3031 bytes) 2024-09-12T07:21:02.712 INFO:teuthology.orchestra.run.smithi046.stdout:5440 read 0x323669b thru 0x3237b36 (0x149c bytes) 2024-09-12T07:21:02.712 INFO:teuthology.orchestra.run.smithi046.stdout:5441 punch from 0x9cc174 to 0x9cfc42, (0x3ace bytes) 2024-09-12T07:21:02.712 INFO:teuthology.orchestra.run.smithi046.stdout:5442 read 0x15e3c65 thru 0x15f31b2 (0xf54e bytes) 2024-09-12T07:21:02.721 INFO:teuthology.orchestra.run.smithi046.stdout:5445 trunc from 0x541ec7f to 0xd4880e9 2024-09-12T07:21:02.726 INFO:teuthology.orchestra.run.smithi046.stdout:5446 write 0xce24b7 thru 0xcebcba (0x9804 bytes) 2024-09-12T07:21:02.729 INFO:teuthology.orchestra.run.smithi046.stdout:5450 writesame 0xb56d8e0 thru 0xb57764b data_size 0x1a(0x9d6c bytes) 2024-09-12T07:21:02.733 INFO:teuthology.orchestra.run.smithi046.stdout:5451 writesame 0x27bc9c9 thru 0x27c4b7a data_size 0xd(0x81b2 bytes) 2024-09-12T07:21:02.734 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:02.734 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:02.737 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:02.737 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:02.737 INFO:teuthology.orchestra.run.smithi046.stdout:5455 punch from 0x3a2e6de to 0x3a30d68, (0x268a bytes) 2024-09-12T07:21:02.737 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:02.737 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:02.737 INFO:teuthology.orchestra.run.smithi046.stdout:5458 write 0x1cd32b1 thru 0x1ce2b98 (0xf8e8 bytes) 2024-09-12T07:21:02.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:02.743 INFO:teuthology.orchestra.run.smithi046.stdout:5459 trunc from 0xd4880e9 to 0xa6be96c 2024-09-12T07:21:02.753 INFO:teuthology.orchestra.run.smithi046.stdout:5460 read 0x2cff28e thru 0x2d08249 (0x8fbc bytes) 2024-09-12T07:21:02.754 INFO:teuthology.orchestra.run.smithi046.stdout:5462 write 0x8163008 thru 0x817116d (0xe166 bytes) 2024-09-12T07:21:02.762 INFO:teuthology.orchestra.run.smithi046.stdout:5463 trunc from 0xa6be96c to 0x69f36f8 2024-09-12T07:21:02.772 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:02.772 INFO:teuthology.orchestra.run.smithi046.stdout:5465 trunc from 0x69f36f8 to 0x6ebf29e 2024-09-12T07:21:02.777 INFO:teuthology.orchestra.run.smithi046.stdout:5469 read 0x59a217f thru 0x59ad394 (0xb216 bytes) 2024-09-12T07:21:02.777 INFO:teuthology.orchestra.run.smithi046.stdout:5470 write 0x2d120e3 thru 0x2d12e35 (0xd53 bytes) 2024-09-12T07:21:02.793 INFO:teuthology.orchestra.run.smithi046.stdout:5472 write 0xcd91b5d thru 0xcd99ae2 (0x7f86 bytes) 2024-09-12T07:21:02.796 INFO:teuthology.orchestra.run.smithi046.stdout:5473 write 0x74a5707 thru 0x74b14f2 (0xbdec bytes) 2024-09-12T07:21:02.799 INFO:teuthology.orchestra.run.smithi046.stdout:5474 read 0x364e873 thru 0x3655f39 (0x76c7 bytes) 2024-09-12T07:21:02.800 INFO:teuthology.orchestra.run.smithi046.stdout:5475 write 0x91b3df6 thru 0x91c1888 (0xda93 bytes) 2024-09-12T07:21:02.804 INFO:teuthology.orchestra.run.smithi046.stdout:5476 writesame 0x6410d00 thru 0x641e431 data_size 0x1(0xd732 bytes) 2024-09-12T07:21:02.826 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:02.826 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:02.828+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:02.835 INFO:teuthology.orchestra.run.smithi046.stdout:5477 write 0xa681768 thru 0xa68b795 (0xa02e bytes) 2024-09-12T07:21:02.838 INFO:teuthology.orchestra.run.smithi046.stdout:5478 write 0x414e250 thru 0x41503c3 (0x2174 bytes) 2024-09-12T07:21:02.841 INFO:teuthology.orchestra.run.smithi046.stdout:5479 read 0x7edd0c2 thru 0x7eec418 (0xf357 bytes) 2024-09-12T07:21:02.846 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:02.854 INFO:teuthology.orchestra.run.smithi046.stdout:5480 trunc from 0xcd99ae3 to 0xe2d56d 2024-09-12T07:21:02.866 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:02.867 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:02.884 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:02.886 INFO:teuthology.orchestra.run.smithi046.stdout:5482 write 0xce51f34 thru 0xce5a198 (0x8265 bytes) 2024-09-12T07:21:02.889 INFO:teuthology.orchestra.run.smithi046.stdout:5485 write 0x712ffe3 thru 0x71324a4 (0x24c2 bytes) 2024-09-12T07:21:02.894 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:02.894 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:02.898 INFO:teuthology.orchestra.run.smithi046.stdout:5486 trunc from 0xce5a199 to 0x7b10c22 2024-09-12T07:21:02.903 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:02.903 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:02.904 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:02.905 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:02.906+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:02.909 INFO:teuthology.orchestra.run.smithi046.stdout:5487 read 0x5e3fa64 thru 0x5e49707 (0x9ca4 bytes) 2024-09-12T07:21:02.909 INFO:teuthology.orchestra.run.smithi046.stdout:5488 trunc from 0x7b10c22 to 0x7e372ee 2024-09-12T07:21:02.913 INFO:teuthology.orchestra.run.smithi046.stdout:5489 punch from 0x4dba305 to 0x4dc837e, (0xe079 bytes) 2024-09-12T07:21:02.921 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:02.921 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:02.922 INFO:teuthology.orchestra.run.smithi046.stdout:5490 write 0xab5e9d8 thru 0xab5f1f3 (0x81c bytes) 2024-09-12T07:21:02.934 INFO:teuthology.orchestra.run.smithi046.stdout:5492 write 0xcd4c6ca thru 0xcd4f7fe (0x3135 bytes) 2024-09-12T07:21:02.940 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:02.941 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:02.950 INFO:teuthology.orchestra.run.smithi046.stdout:5493 trunc from 0xcd4f7ff to 0x125dded 2024-09-12T07:21:02.961 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:02.962 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:02.962 INFO:teuthology.orchestra.run.smithi046.stdout:5494 punch from 0xe255c to 0xe36cc, (0x1170 bytes) 2024-09-12T07:21:02.962 INFO:teuthology.orchestra.run.smithi046.stdout:5495 write 0x5849ea thru 0x590927 (0xbf3e bytes) 2024-09-12T07:21:02.963 INFO:teuthology.orchestra.run.smithi046.stdout:5496 punch from 0x11105e9 to 0x11196b8, (0x90cf bytes) 2024-09-12T07:21:02.963 INFO:teuthology.orchestra.run.smithi046.stdout:5497 punch from 0x5605b1 to 0x56ed7b, (0xe7ca bytes) 2024-09-12T07:21:02.963 INFO:teuthology.orchestra.run.smithi046.stdout:5498 punch from 0x9558b3 to 0x961683, (0xbdd0 bytes) 2024-09-12T07:21:02.970 INFO:teuthology.orchestra.run.smithi046.stdout:5499 write 0x2203521 thru 0x2209ceb (0x67cb bytes) 2024-09-12T07:21:02.990 INFO:teuthology.orchestra.run.smithi046.stdout:5500 write 0xbe1071f thru 0xbe1c9ca (0xc2ac bytes) 2024-09-12T07:21:02.993 INFO:teuthology.orchestra.run.smithi046.stdout:5501 write 0x8bbfb95 thru 0x8bc7f1f (0x838b bytes) 2024-09-12T07:21:02.996 INFO:teuthology.orchestra.run.smithi046.stdout:5502 read 0x7fbbf38 thru 0x7fbe8c7 (0x2990 bytes) 2024-09-12T07:21:03.006 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:03.007+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:03.006 INFO:teuthology.orchestra.run.smithi046.stdout:5503 trunc from 0xbe1c9cb to 0x34e8cfc 2024-09-12T07:21:03.015 INFO:teuthology.orchestra.run.smithi046.stdout:5504 read 0x235d7d4 thru 0x2362259 (0x4a86 bytes) 2024-09-12T07:21:03.016 INFO:teuthology.orchestra.run.smithi046.stdout:5506 clone 43 order 24 su 524288 sc 10 2024-09-12T07:21:03.053 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:03.054 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:03.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:03.107 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:03.108+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:03.168 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:03.168 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:03.177 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:03.220 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:03.222+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:03.285 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 70.756531585999994, 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:21:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:21:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.904733919999998, 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.904610642999998, 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:21:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:21:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:21:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:21:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.904407251999999, 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:21:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:21:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:21:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.904271424999997, 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:21:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:21:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:21:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.903944768999999, 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:21:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:21:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:21:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:21:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:21:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:21:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.903905881, 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:21:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:21:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.903539006999999, 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:21:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.903404150999997, 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.903034259999998, 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:21:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.902844062, 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:21:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.902623982999998, 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:21:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.902480318000002, 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:21:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.902254145999997, 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:21:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.902053788000003, 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:21:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:21:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.901645905000002, 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:21:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:21:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:21:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.901180023000002, 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:21:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:21:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.899577813000001, 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:21:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.899043777999999, 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:21:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:21:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.642845708000003, 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:21:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:21:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:21:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:21:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.378 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:03.380 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:03.380 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:03.381+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:03.446 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7, 3] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:21:03.446 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:21:03.447 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 2, 4, 5, 6, 7, 3] 2024-09-12T07:21:03.447 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-12T07:21:03.491 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:03.493+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:03.531 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:03.532 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:03.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:03.581 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:03.583+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:03.597 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-12T07:21:03.603 INFO:teuthology.orchestra.run.smithi046.stdout:5507 read 0xce0611 thru 0xce43a1 (0x3d91 bytes) 2024-09-12T07:21:03.605 INFO:teuthology.orchestra.run.smithi046.stdout:5508 read 0xf875ad thru 0xf94aa0 (0xd4f4 bytes) 2024-09-12T07:21:03.607 INFO:teuthology.orchestra.run.smithi046.stdout:5509 read 0x2385761 thru 0x238d529 (0x7dc9 bytes) 2024-09-12T07:21:03.608 INFO:teuthology.orchestra.run.smithi046.stdout:5510 writesame 0x17b3a1d thru 0x17be239 data_size 0x1(0xa81d bytes) 2024-09-12T07:21:03.648 INFO:teuthology.orchestra.run.smithi046.stdout:5512 write 0xc224d34 thru 0xc22ecb2 (0x9f7f bytes) 2024-09-12T07:21:03.649 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:03.649 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:03.651 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:03.651 INFO:teuthology.orchestra.run.smithi046.stdout:5516 read 0x65ae822 thru 0x65b2c45 (0x4424 bytes) 2024-09-12T07:21:03.652 INFO:teuthology.orchestra.run.smithi046.stdout:5517 punch from 0x10ae60d to 0x10bbc1c, (0xd60f bytes) 2024-09-12T07:21:03.652 INFO:teuthology.orchestra.run.smithi046.stdout:5518 write 0x166cfdd thru 0x167623f (0x9263 bytes) 2024-09-12T07:21:03.655 INFO:teuthology.orchestra.run.smithi046.stdout:5519 write 0x1a923ea thru 0x1aa0d8c (0xe9a3 bytes) 2024-09-12T07:21:03.658 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:03.661 INFO:teuthology.orchestra.run.smithi046.stdout:5520 read 0xb14040d thru 0xb14ce3c (0xca30 bytes) 2024-09-12T07:21:03.667 INFO:teuthology.orchestra.run.smithi046.stdout:5521 write 0xcb6dbfe thru 0xcb7b693 (0xda96 bytes) 2024-09-12T07:21:03.676 INFO:teuthology.orchestra.run.smithi046.stdout:5522 write 0xd7cb7d2 thru 0xd7cc80d (0x103c bytes) 2024-09-12T07:21:03.678 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:03.679 INFO:teuthology.orchestra.run.smithi046.stdout:5525 writesame 0x6d3178e thru 0x6d35f3d data_size 0x4a(0x47b0 bytes) 2024-09-12T07:21:03.682 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:03.683+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:03.691 INFO:teuthology.orchestra.run.smithi046.stdout:5526 writesame 0xe32f87c thru 0xe33cd66 data_size 0x1(0xd4eb bytes) 2024-09-12T07:21:03.691 INFO:teuthology.orchestra.run.smithi046.stdout:5527 write 0x803fb0e thru 0x8045c35 (0x6128 bytes) 2024-09-12T07:21:03.705 INFO:teuthology.orchestra.run.smithi046.stdout:5528 trunc from 0xe33cd67 to 0x4b64f7d 2024-09-12T07:21:03.730 INFO:teuthology.orchestra.run.smithi046.stdout:5529 read 0x26793b4 thru 0x2681724 (0x8371 bytes) 2024-09-12T07:21:03.730 INFO:teuthology.orchestra.run.smithi046.stdout:5531 write 0x15cdf79 thru 0x15daa02 (0xca8a bytes) 2024-09-12T07:21:03.731 INFO:teuthology.orchestra.run.smithi046.stdout:5532 read 0x4413a7d thru 0x4420abb (0xd03f bytes) 2024-09-12T07:21:03.739 INFO:teuthology.orchestra.run.smithi046.stdout:5533 writesame 0x6b848ab thru 0x6b89856 data_size 0x2(0x4fac bytes) 2024-09-12T07:21:03.740 INFO:teuthology.orchestra.run.smithi046.stdout:5534 punch from 0x3943054 to 0x3950237, (0xd1e3 bytes) 2024-09-12T07:21:03.743 INFO:teuthology.orchestra.run.smithi046.stdout:5536 trunc from 0x6b89857 to 0x8748dc9 2024-09-12T07:21:03.755 INFO:teuthology.orchestra.run.smithi046.stdout:5537 write 0x98a3b8b thru 0x98ab443 (0x78b9 bytes) 2024-09-12T07:21:03.761 INFO:teuthology.orchestra.run.smithi046.stdout:5538 writesame 0x9c2df88 thru 0x9c3d4de data_size 0x1(0xf557 bytes) 2024-09-12T07:21:03.763 INFO:teuthology.orchestra.run.smithi046.stdout:5539 trunc from 0x9c3d4df to 0xa2ca666 2024-09-12T07:21:03.775 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227504371000002, 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:21:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:21:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227483833999997, 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:21:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227463831000001, 2024-09-12T07:21:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:21:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227443882000003, 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227423014000003, 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:21:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:21:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227401313000001, 2024-09-12T07:21:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:21:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:21:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:21:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227380873000001, 2024-09-12T07:21:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:21:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:21:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:21:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:21:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:21:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:21:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:21:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:21:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227353151999999, 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:21:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:21:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:21:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:21:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:21:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:21:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227330217000002, 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:21:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:21:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:21:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:21:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:21:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:21:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227309916999999, 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:21:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:21:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:21:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227289640999999, 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:21:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:21:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:21:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227269481999997, 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227132284, 2024-09-12T07:21:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:21:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:21:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:21:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.227109636000002, 2024-09-12T07:21:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:21:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:21:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:21:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:21:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:21:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68088 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 393216~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.220666876999999, 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10346986399999999, 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68088 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555923+0000", 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.891e-06 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555929+0000", 2024-09-12T07:21:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0549999999999996e-06 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555934+0000", 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6419999999999998e-06 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659060+0000", 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103125862 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659139+0000", 2024-09-12T07:21:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9450000000000007e-05 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659391+0000", 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025158400000000001 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.220647649999997, 2024-09-12T07:21:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:21:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.220418013, 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:21:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:21:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:21:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:21:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.213494322000003, 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.213134738999997, 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:21:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.213054753000002, 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:03.867 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:03.869 INFO:teuthology.orchestra.run.smithi046.stdout:5540 write 0xba8deb8 thru 0xba95dde (0x7f27 bytes) 2024-09-12T07:21:03.869 INFO:teuthology.orchestra.run.smithi046.stdout:5541 write 0x4aa9973 thru 0x4aae577 (0x4c05 bytes) 2024-09-12T07:21:03.870 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:03.870 INFO:teuthology.orchestra.run.smithi046.stdout:5543 trunc from 0xba95ddf to 0x22f617e 2024-09-12T07:21:03.870 INFO:teuthology.orchestra.run.smithi046.stdout:5544 read 0xe839ef thru 0xe90eef (0xd501 bytes) 2024-09-12T07:21:03.870 INFO:teuthology.orchestra.run.smithi046.stdout:5546 write 0x89ff9d thru 0x8aee7b (0xeedf bytes) 2024-09-12T07:21:03.870 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-12T07:21:03.808+0000 7f0757424640 -1 mon.a@0(leader).osd e102 definitely_dead 0 2024-09-12T07:21:03.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:03.872 INFO:teuthology.orchestra.run.smithi046.stdout:5547 read 0x1952441 thru 0x1957f75 (0x5b35 bytes) 2024-09-12T07:21:03.872 INFO:teuthology.orchestra.run.smithi046.stdout:5548 read 0x81b519 thru 0x81d141 (0x1c29 bytes) 2024-09-12T07:21:03.872 INFO:teuthology.orchestra.run.smithi046.stdout:5549 write 0xb194265 thru 0xb19f593 (0xb32f bytes) 2024-09-12T07:21:03.872 INFO:teuthology.orchestra.run.smithi046.stdout:5550 punch from 0x7faea55 to 0x7fbd972, (0xef1d bytes) 2024-09-12T07:21:03.872 INFO:teuthology.orchestra.run.smithi046.stdout:5551 writesame 0x9e6d1c2 thru 0x9e76ed4 data_size 0x1(0x9d13 bytes) 2024-09-12T07:21:03.872 INFO:teuthology.orchestra.run.smithi046.stdout:5555 write 0xa84db9d thru 0xa85d3f6 (0xf85a bytes) 2024-09-12T07:21:03.872 INFO:teuthology.orchestra.run.smithi046.stdout:5557 writesame 0x677a366 thru 0x677d339 data_size 0x2(0x2fd4 bytes) 2024-09-12T07:21:03.872 INFO:teuthology.orchestra.run.smithi046.stdout:5558 writesame 0x746d241 thru 0x747b760 data_size 0x9c(0xe520 bytes) 2024-09-12T07:21:03.872 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:03.873 INFO:teuthology.orchestra.run.smithi046.stdout:5560 write 0x1c714ff thru 0x1c7e20b (0xcd0d bytes) 2024-09-12T07:21:03.873 INFO:teuthology.orchestra.run.smithi046.stdout:5562 trunc from 0xb19f594 to 0x34c4ac6 2024-09-12T07:21:03.873 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:03.870+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:03.879 INFO:teuthology.orchestra.run.smithi046.stdout:5563 write 0x2eca750 thru 0x2ed581d (0xb0ce bytes) 2024-09-12T07:21:03.880 INFO:teuthology.orchestra.run.smithi046.stdout:5564 write 0x15a4a99 thru 0x15b1c0f (0xd177 bytes) 2024-09-12T07:21:03.891 INFO:teuthology.orchestra.run.smithi046.stdout:5565 writesame 0x612366b thru 0x612d6ed data_size 0x1(0xa083 bytes) 2024-09-12T07:21:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:5566 punch from 0xf57d7a to 0xf5864e, (0x8d4 bytes) 2024-09-12T07:21:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:5568 write 0x1ebd9ab thru 0x1ec17e0 (0x3e36 bytes) 2024-09-12T07:21:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:03.897 INFO:teuthology.orchestra.run.smithi046.stdout:5572 trunc from 0x612d6ee to 0x368d84c 2024-09-12T07:21:03.909 INFO:teuthology.orchestra.run.smithi046.stdout:5573 trunc from 0x368d84c to 0x5d45ae8 2024-09-12T07:21:03.915 INFO:teuthology.orchestra.run.smithi046.stdout:5574 read 0xf79fa9 thru 0xf8533e (0xb396 bytes) 2024-09-12T07:21:03.927 INFO:teuthology.orchestra.run.smithi046.stdout:5576 write 0xab1c6ad thru 0xab1ce82 (0x7d6 bytes) 2024-09-12T07:21:03.930 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:03.930 INFO:teuthology.orchestra.run.smithi046.stdout:5579 read 0x82e7be6 thru 0x82f2421 (0xa83c bytes) 2024-09-12T07:21:03.935 INFO:teuthology.orchestra.run.smithi046.stdout:5581 write 0xabee448 thru 0xabf66b0 (0x8269 bytes) 2024-09-12T07:21:03.939 INFO:teuthology.orchestra.run.smithi046.stdout:5582 writesame 0x767502d thru 0x7683605 data_size 0x1(0xe5d9 bytes) 2024-09-12T07:21:03.968 INFO:teuthology.orchestra.run.smithi046.stdout:5583 trunc from 0xabf66b1 to 0x1cce72d 2024-09-12T07:21:03.969 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:03.971+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:03.980 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:03.992 INFO:teuthology.orchestra.run.smithi046.stdout:5585 write 0x7db543b thru 0x7dbf1b2 (0x9d78 bytes) 2024-09-12T07:21:03.993 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:03.994 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:03.994 INFO:teuthology.orchestra.run.smithi046.stdout:5591 write 0x356113e thru 0x356704c (0x5f0f bytes) 2024-09-12T07:21:03.996 INFO:teuthology.orchestra.run.smithi046.stdout:5593 trunc from 0x7dbf1b3 to 0x6f74260 2024-09-12T07:21:03.998 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "pg_update_log_missing_reply(3.1b epoch 102/24 rep_tid 2601 lcod 102'321)", 2024-09-12T07:21:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:04.000949+0000", 2024-09-12T07:21:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.7117000000000001e-05, 2024-09-12T07:21:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0025999999999999e-05, 2024-09-12T07:21:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:21:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:04.000949+0000", 2024-09-12T07:21:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:04.000949+0000", 2024-09-12T07:21:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:04.000949+0000", 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8899999999999998e-07 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:04.000953+0000", 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4699999999999998e-06 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:04.000964+0000", 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0732e-05 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:04.000973+0000", 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7860000000000008e-06 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "pg_update_log_missing_reply(3.6 epoch 102/100 rep_tid 2603 lcod 102'475)", 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:04.000993+0000", 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9999828, 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3472000000000002e-05, 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.002 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:04.000993+0000", 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:04.000993+0000", 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:04.000994+0000", 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7800000000000002e-07 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:04.000996+0000", 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:04.001001+0000", 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1780000000000002e-06 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:04.001005+0000", 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0670000000000002e-06 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:165090 3.1 3:839a2328:::rbd_data.11606fa8d72f.0000000000000000:head [stat out=16b,zero 12058624~1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:03.999175+0000", 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0018004150000000001, 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0018838450000000001, 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 165090 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999175+0000", 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999175+0000", 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999179+0000", 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1750000000000001e-06 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999191+0000", 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2696e-05 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999218+0000", 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6621000000000001e-05 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999231+0000", 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.272e-05 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999304+0000", 2024-09-12T07:21:04.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2990000000000004e-05 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999384+0000", 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0111999999999995e-05 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:165097 3.1 3:84e98d57:::rbd_data.11606fa8d72f.0000000000000007:head [stat out=16b,zero 11534336~1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:03.999371+0000", 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.001604797, 2024-09-12T07:21:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001724384, 2024-09-12T07:21:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 165097 2024-09-12T07:21:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999371+0000", 2024-09-12T07:21:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999371+0000", 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999371+0000", 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4499999999999997e-07 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999374+0000", 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1770000000000001e-06 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999380+0000", 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999385+0000", 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3170000000000001e-06 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999808+0000", 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00042278599999999997 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:03.999894+0000", 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5453999999999999e-05 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:04.014 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 4 2024-09-12T07:21:04.015 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:04.015 INFO:teuthology.orchestra.run.smithi046.stdout:5594 trunc from 0x6f74260 to 0xff0106 2024-09-12T07:21:04.016 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:04.019 INFO:teuthology.orchestra.run.smithi046.stdout:5596 punch from 0x7ccb6d to 0x7cec03, (0x2096 bytes) 2024-09-12T07:21:04.019 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:04.019 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:04.026 INFO:teuthology.orchestra.run.smithi046.stdout:5599 writesame 0x2f30207 thru 0x2f37ede data_size 0x44(0x7cd8 bytes) 2024-09-12T07:21:04.027 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:04.031 INFO:teuthology.orchestra.run.smithi046.stdout:5602 trunc from 0x2f37edf to 0x5ec713f 2024-09-12T07:21:04.036 INFO:teuthology.orchestra.run.smithi046.stdout:5603 writesame 0x5b129c8 thru 0x5b14617 data_size 0x18(0x1c50 bytes) 2024-09-12T07:21:04.038 INFO:teuthology.orchestra.run.smithi046.stdout:5604 read 0x4288f11 thru 0x4296544 (0xd634 bytes) 2024-09-12T07:21:04.038 INFO:teuthology.orchestra.run.smithi046.stdout:5605 writesame 0x5ccd585 thru 0x5cd928c data_size 0x5c(0xbd08 bytes) 2024-09-12T07:21:04.047 INFO:teuthology.orchestra.run.smithi046.stdout:5607 writesame 0x731415d thru 0x7320a75 data_size 0x1(0xc919 bytes) 2024-09-12T07:21:04.049 INFO:teuthology.orchestra.run.smithi046.stdout:5609 read 0x594b7da thru 0x5958018 (0xc83f bytes) 2024-09-12T07:21:04.049 INFO:teuthology.orchestra.run.smithi046.stdout:5610 writesame 0x40ddcec thru 0x40e6ac1 data_size 0x5(0x8dd6 bytes) 2024-09-12T07:21:04.053 INFO:teuthology.orchestra.run.smithi046.stdout:5611 read 0x1a6cd7c thru 0x1a6d128 (0x3ad bytes) 2024-09-12T07:21:04.062 INFO:teuthology.orchestra.run.smithi046.stdout:5613 trunc from 0x7320a76 to 0xeb23c3a 2024-09-12T07:21:04.068 INFO:teuthology.orchestra.run.smithi046.stdout:5614 read 0x2112112 thru 0x2115b9a (0x3a89 bytes) 2024-09-12T07:21:04.068 INFO:teuthology.orchestra.run.smithi046.stdout:5615 write 0xc6aadfa thru 0xc6af838 (0x4a3f bytes) 2024-09-12T07:21:04.069 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:04.071+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:04.071 INFO:teuthology.orchestra.run.smithi046.stdout:5616 punch from 0xbc8ed48 to 0xbc9a4ac, (0xb764 bytes) 2024-09-12T07:21:04.072 INFO:teuthology.orchestra.run.smithi046.stdout:5618 read 0xc02c1b2 thru 0xc031096 (0x4ee5 bytes) 2024-09-12T07:21:04.072 INFO:teuthology.orchestra.run.smithi046.stdout:5619 punch from 0x73a8f48 to 0x73af456, (0x650e bytes) 2024-09-12T07:21:04.072 INFO:teuthology.orchestra.run.smithi046.stdout:5620 read 0xeadb2ec thru 0xeade4da (0x31ef bytes) 2024-09-12T07:21:04.072 INFO:teuthology.orchestra.run.smithi046.stdout:5621 write 0xb9d80c1 thru 0xb9dc938 (0x4878 bytes) 2024-09-12T07:21:04.074 INFO:teuthology.orchestra.run.smithi046.stdout:5622 writesame 0xb9a0a6d thru 0xb9a16a8 data_size 0x24(0xc3c bytes) 2024-09-12T07:21:04.075 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:04.075 INFO:teuthology.orchestra.run.smithi046.stdout:5624 read 0x3bfd88f thru 0x3bfe664 (0xdd6 bytes) 2024-09-12T07:21:04.076 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:04.076 INFO:teuthology.orchestra.run.smithi046.stdout:5627 read 0x9840f73 thru 0x9846dbe (0x5e4c bytes) 2024-09-12T07:21:04.077 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:04.077 INFO:teuthology.orchestra.run.smithi046.stdout:5631 punch from 0xda17707 to 0xda185d6, (0xecf bytes) 2024-09-12T07:21:04.077 INFO:teuthology.orchestra.run.smithi046.stdout:5632 read 0x4563f45 thru 0x456b797 (0x7853 bytes) 2024-09-12T07:21:04.077 INFO:teuthology.orchestra.run.smithi046.stdout:5634 read 0x80b927d thru 0x80bb266 (0x1fea bytes) 2024-09-12T07:21:04.077 INFO:teuthology.orchestra.run.smithi046.stdout:5635 write 0xaeeb3c thru 0xaf4a11 (0x5ed6 bytes) 2024-09-12T07:21:04.078 INFO:teuthology.orchestra.run.smithi046.stdout:5636 writesame 0x13064da thru 0x131521c data_size 0x1(0xed43 bytes) 2024-09-12T07:21:04.104 INFO:teuthology.orchestra.run.smithi046.stdout:5639 writesame 0x16432ef thru 0x1649fba data_size 0x2c(0x6ccc bytes) 2024-09-12T07:21:04.106 INFO:teuthology.orchestra.run.smithi046.stdout:5640 read 0xb0260ea thru 0xb02c5b8 (0x64cf bytes) 2024-09-12T07:21:04.106 INFO:teuthology.orchestra.run.smithi046.stdout:5641 read 0x557d316 thru 0x55819a6 (0x4691 bytes) 2024-09-12T07:21:04.107 INFO:teuthology.orchestra.run.smithi046.stdout:5642 read 0xda6ea2e thru 0xda73626 (0x4bf9 bytes) 2024-09-12T07:21:04.107 INFO:teuthology.orchestra.run.smithi046.stdout:5643 read 0xe8d65a7 thru 0xe8e2a22 (0xc47c bytes) 2024-09-12T07:21:04.107 INFO:teuthology.orchestra.run.smithi046.stdout:5644 punch from 0x2755641 to 0x2762be3, (0xd5a2 bytes) 2024-09-12T07:21:04.107 INFO:teuthology.orchestra.run.smithi046.stdout:5645 punch from 0xc16279d to 0xc172064, (0xf8c7 bytes) 2024-09-12T07:21:04.107 INFO:teuthology.orchestra.run.smithi046.stdout:5646 write 0x4ee7de thru 0x4f99f0 (0xb213 bytes) 2024-09-12T07:21:04.108 INFO:teuthology.orchestra.run.smithi046.stdout:5647 writesame 0x4eab306 thru 0x4eb457d data_size 0x56(0x9278 bytes) 2024-09-12T07:21:04.109 INFO:teuthology.orchestra.run.smithi046.stdout:5648 read 0x3708b2 thru 0x37ae38 (0xa587 bytes) 2024-09-12T07:21:04.110 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:04.110 INFO:teuthology.orchestra.run.smithi046.stdout:5650 write 0xb11aa35 thru 0xb11b43d (0xa09 bytes) 2024-09-12T07:21:04.113 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:04.113 INFO:teuthology.orchestra.run.smithi046.stdout:5652 write 0xc1a05b5 thru 0xc1ad6a4 (0xd0f0 bytes) 2024-09-12T07:21:04.115 INFO:teuthology.orchestra.run.smithi046.stdout:5653 trunc from 0xeb23c3a to 0xd4f6243 2024-09-12T07:21:04.122 INFO:teuthology.orchestra.run.smithi046.stdout:5654 write 0x391a2f4 thru 0x391b5df (0x12ec bytes) 2024-09-12T07:21:04.123 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:04.123 INFO:teuthology.orchestra.run.smithi046.stdout:5659 writesame 0x865904c thru 0x8662c5c data_size 0x1(0x9c11 bytes) 2024-09-12T07:21:04.137 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:04.138 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:04.144 INFO:teuthology.orchestra.run.smithi046.stdout:5660 writesame 0x59eadad thru 0x59ef835 data_size 0x1(0x4a89 bytes) 2024-09-12T07:21:04.146 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:04.154 INFO:teuthology.orchestra.run.smithi046.stdout:5661 write 0x5520af8 thru 0x5525540 (0x4a49 bytes) 2024-09-12T07:21:04.156 INFO:teuthology.orchestra.run.smithi046.stdout:5662 read 0xa71830b thru 0xa71d538 (0x522e bytes) 2024-09-12T07:21:04.157 INFO:teuthology.orchestra.run.smithi046.stdout:5663 write 0x3b3c377 thru 0x3b420c1 (0x5d4b bytes) 2024-09-12T07:21:04.158 INFO:teuthology.orchestra.run.smithi046.stdout:5664 read 0x55c3f19 thru 0x55cb2b3 (0x739b bytes) 2024-09-12T07:21:04.158 INFO:teuthology.orchestra.run.smithi046.stdout:5666 writesame 0x43caca8 thru 0x43ce6c4 data_size 0x13(0x3a1d bytes) 2024-09-12T07:21:04.170 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:04.172+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:04.174 INFO:teuthology.orchestra.run.smithi046.stdout:5667 trunc from 0xd4f6243 to 0xaa8066 2024-09-12T07:21:04.191 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:04.210 INFO:teuthology.orchestra.run.smithi046.stdout:5669 write 0x9d108d0 thru 0x9d2039a (0xfacb bytes) 2024-09-12T07:21:04.212 INFO:teuthology.orchestra.run.smithi046.stdout:5671 read 0x5da250e thru 0x5daf4a8 (0xcf9b bytes) 2024-09-12T07:21:04.222 INFO:teuthology.orchestra.run.smithi046.stdout:5672 writesame 0xd138ca2 thru 0xd146310 data_size 0x1(0xd66f bytes) 2024-09-12T07:21:04.233 INFO:teuthology.orchestra.run.smithi046.stdout:5673 trunc from 0xd146311 to 0x4a42115 2024-09-12T07:21:04.244 INFO:teuthology.orchestra.run.smithi046.stdout:5674 trunc from 0x4a42115 to 0x245b1dd 2024-09-12T07:21:04.254 INFO:teuthology.orchestra.run.smithi046.stdout:5675 trunc from 0x245b1dd to 0x416d97b 2024-09-12T07:21:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:5677 trunc from 0x416d97b to 0x7a7ed68 2024-09-12T07:21:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:5678 read 0x1aa0a42 thru 0x1aa3109 (0x26c8 bytes) 2024-09-12T07:21:04.271 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:04.273+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:04.272 INFO:teuthology.orchestra.run.smithi046.stdout:5680 write 0x7e9807e thru 0x7ea0c5f (0x8be2 bytes) 2024-09-12T07:21:04.275 INFO:teuthology.orchestra.run.smithi046.stdout:5681 read 0xaf6435 thru 0xb03544 (0xd110 bytes) 2024-09-12T07:21:04.275 INFO:teuthology.orchestra.run.smithi046.stdout:5682 read 0x5fe2ab4 thru 0x5febfe9 (0x9536 bytes) 2024-09-12T07:21:04.275 INFO:teuthology.orchestra.run.smithi046.stdout:5683 read 0x66d68f6 thru 0x66dccb0 (0x63bb bytes) 2024-09-12T07:21:04.287 INFO:teuthology.orchestra.run.smithi046.stdout:5684 write 0xd48ef8e thru 0xd497148 (0x81bb bytes) 2024-09-12T07:21:04.289 INFO:teuthology.orchestra.run.smithi046.stdout:5685 punch from 0x579a4b3 to 0x579f479, (0x4fc6 bytes) 2024-09-12T07:21:04.290 INFO:teuthology.orchestra.run.smithi046.stdout:5688 write 0xb14d47e thru 0xb14f380 (0x1f03 bytes) 2024-09-12T07:21:04.292 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:04.292 INFO:teuthology.orchestra.run.smithi046.stdout:5691 punch from 0x1dc5845 to 0x1dd3c26, (0xe3e1 bytes) 2024-09-12T07:21:04.292 INFO:teuthology.orchestra.run.smithi046.stdout:5694 writesame 0xfa156d thru 0xfb018b data_size 0x1(0xec1f bytes) 2024-09-12T07:21:04.301 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:04.301 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82749 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 429056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.658114243999997, 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055052166, 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.302 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82749 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0800000000000001e-07 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644459+0000", 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1769999999999999e-06 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644462+0000", 2024-09-12T07:21:04.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8779999999999998e-06 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644465+0000", 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6509999999999999e-06 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699279+0000", 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054813832999999999 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699359+0000", 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0150999999999999e-05 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699510+0000", 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015026800000000001 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82750 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 432128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:21:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.658101360000003, 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055234722, 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82750 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644471+0000", 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.17e-07 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644472+0000", 2024-09-12T07:21:04.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.19e-06 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644482+0000", 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0332e-05 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644489+0000", 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7039999999999997e-06 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699549+0000", 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055060588000000001 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699592+0000", 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2707000000000003e-05 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699705+0000", 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112984 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82751 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 436224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.658074006, 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055415691000000003, 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82751 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.311 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644846+0000", 2024-09-12T07:21:04.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034807600000000001 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644850+0000", 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1649999999999999e-06 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644860+0000", 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6949999999999995e-06 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644866+0000", 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3640000000000004e-06 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699735+0000", 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054868888999999997 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699776+0000", 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0722000000000001e-05 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699913+0000", 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013778 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82752 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 439296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.657696262999998, 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226917, 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82752 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644878+0000", 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7850000000000001e-06 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644882+0000", 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9990000000000002e-06 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644885+0000", 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.965e-06 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699952+0000", 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067243000000002 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699989+0000", 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7450999999999998e-05 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700102+0000", 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113124 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82753 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 443392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:21:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.657683607000003, 2024-09-12T07:21:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055422102000000001, 2024-09-12T07:21:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82753 2024-09-12T07:21:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.320 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:21:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:21:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645016+0000", 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000127847 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645024+0000", 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7419999999999997e-06 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645030+0000", 2024-09-12T07:21:04.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7229999999999998e-06 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645034+0000", 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9380000000000002e-06 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700132+0000", 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055097699 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700171+0000", 2024-09-12T07:21:04.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9093e-05 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700310+0000", 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013905999999999999 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82754 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 446464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:21:04.324 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.657527700000003, 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055424703999999998, 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82754 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645318+0000", 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027397499999999998 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645322+0000", 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.766e-06 2024-09-12T07:21:04.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645331+0000", 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2140000000000002e-06 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645338+0000", 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0929999999999997e-06 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700330+0000", 2024-09-12T07:21:04.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054991384999999997 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700369+0000", 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700469+0000", 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9913000000000004e-05 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82755 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 449536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.657222709999999, 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055388918000000002, 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82755 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.329 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645500+0000", 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015050999999999999 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645502+0000", 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645509+0000", 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3969999999999999e-06 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645513+0000", 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9230000000000004e-06 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700507+0000", 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054994692999999997 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700583+0000", 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5656999999999997e-05 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700738+0000", 2024-09-12T07:21:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154869 2024-09-12T07:21:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82756 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 453632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:21:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.657051822, 2024-09-12T07:21:04.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055451746000000003, 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82756 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645763+0000", 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000243105 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645766+0000", 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0750000000000002e-06 2024-09-12T07:21:04.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645774+0000", 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9370000000000004e-06 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645783+0000", 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5159999999999994e-06 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700780+0000", 2024-09-12T07:21:04.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054997329999999997 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700835+0000", 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4789e-05 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700972+0000", 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013699400000000001 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82757 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.656777454, 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055382590000000002, 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82757 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645926+0000", 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131979 2024-09-12T07:21:04.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645929+0000", 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0199999999999999e-06 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645935+0000", 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645939+0000", 2024-09-12T07:21:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.924e-06 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701007+0000", 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067374000000002 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701043+0000", 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6325999999999997e-05 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701177+0000", 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013387300000000001 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82758 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 460800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.656620060999998, 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055417890999999997, 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82758 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:21:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646164+0000", 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000212189 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646167+0000", 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646173+0000", 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4139999999999999e-06 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646177+0000", 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701208+0000", 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055031322000000001 2024-09-12T07:21:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701254+0000", 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5701999999999997e-05 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701370+0000", 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115456 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82759 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 463872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.656387500000001, 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055376838999999997, 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82759 2024-09-12T07:21:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646368+0000", 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183399 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646372+0000", 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9190000000000001e-06 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646380+0000", 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7819999999999999e-06 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646384+0000", 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701407+0000", 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055022763000000002 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701455+0000", 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7660000000000001e-05 2024-09-12T07:21:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701561+0000", 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010642899999999999 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82760 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.656179950999999, 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055355572999999998, 2024-09-12T07:21:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82760 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646631+0000", 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000238798 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646634+0000", 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.016e-06 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646640+0000", 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2750000000000001e-06 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646644+0000", 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7859999999999998e-06 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701599+0000", 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054955693999999999 2024-09-12T07:21:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701638+0000", 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8319999999999999e-05 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701747+0000", 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109684 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82761 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.655921321999998, 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055296990999999997, 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82761 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646756+0000", 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105899 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646759+0000", 2024-09-12T07:21:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8380000000000002e-06 2024-09-12T07:21:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646765+0000", 2024-09-12T07:21:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:21:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646769+0000", 2024-09-12T07:21:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7900000000000001e-06 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701785+0000", 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055015888999999998 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701832+0000", 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6514000000000002e-05 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701947+0000", 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115833 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82762 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 474112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.655795755, 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055346863000000003, 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82762 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647063+0000", 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000287115 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647066+0000", 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0189999999999998e-06 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647075+0000", 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7099999999999996e-06 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647081+0000", 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6790000000000002e-06 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701977+0000", 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054896665999999997 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702015+0000", 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.79e-05 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702123+0000", 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107774 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82763 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.655480838000003, 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226509999999999, 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82763 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647236+0000", 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000144597 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647239+0000", 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1329999999999999e-06 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647245+0000", 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647249+0000", 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2270000000000004e-06 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702160+0000", 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054910549000000003 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702205+0000", 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5575000000000003e-05 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702317+0000", 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011194899999999999 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82764 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 481280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.655315086999998, 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055229250000000001, 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82764 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647471+0000", 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000214625 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647474+0000", 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8509999999999999e-06 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647480+0000", 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.28e-06 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647484+0000", 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.771e-06 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702346+0000", 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054861551000000001 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702382+0000", 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.65e-05 2024-09-12T07:21:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702486+0000", 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010367200000000001 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82765 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 485376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.655079540000003, 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055184729000000002, 2024-09-12T07:21:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82765 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647681+0000", 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188965 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647693+0000", 2024-09-12T07:21:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.225e-05 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647700+0000", 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9929999999999998e-06 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647704+0000", 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9469999999999996e-06 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702522+0000", 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054817346000000003 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702566+0000", 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4672999999999997e-05 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702677+0000", 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110555 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82766 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 488448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.654860302000003, 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055132717999999997, 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82766 2024-09-12T07:21:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:21:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:21:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647959+0000", 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024739599999999999 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647964+0000", 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9409999999999998e-06 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647975+0000", 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1404e-05 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647983+0000", 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2969999999999998e-06 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702706+0000", 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054723690999999998 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702743+0000", 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6464999999999997e-05 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702844+0000", 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101524 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82767 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 492544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.654576261000003, 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055054448999999998, 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82767 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648182+0000", 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018652299999999999 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648185+0000", 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3289999999999998e-06 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648192+0000", 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7159999999999997e-06 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648196+0000", 2024-09-12T07:21:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1439999999999996e-06 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702873+0000", 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054676356000000002 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702930+0000", 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7643999999999999e-05 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703050+0000", 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119737 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:148945 3.10 3:0cac0a45:::rbd_data.1160161ce0c8.000000000000009d:head [delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.868770300000001, 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89878390500000005, 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 148945 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433803+0000", 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.164e-06 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433807+0000", 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5000000000000001e-06 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433816+0000", 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1859999999999998e-06 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433822+0000", 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5300000000000004e-06 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433886+0000", 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4510000000000004e-05 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433896+0000", 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0012e-05 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329359+0000", 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89546257399999996 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329585+0000", 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022641800000000001 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329712+0000", 2024-09-12T07:21:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126339 2024-09-12T07:21:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.330704+0000", 2024-09-12T07:21:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099219900000000003 2024-09-12T07:21:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332513+0000", 2024-09-12T07:21:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001808689 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332544+0000", 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1186999999999998e-05 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332585+0000", 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1597000000000002e-05 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.394 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:04.395 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:04.396 INFO:teuthology.orchestra.run.smithi046.stdout:5695 trunc from 0xd497149 to 0x667aae1 2024-09-12T07:21:04.396 INFO:teuthology.orchestra.run.smithi046.stdout:5696 write 0x8c7c64c thru 0x8c7fc77 (0x362c bytes) 2024-09-12T07:21:04.396 INFO:teuthology.orchestra.run.smithi046.stdout:5697 trunc from 0x8c7fc78 to 0x5e8c3d2 2024-09-12T07:21:04.396 INFO:teuthology.orchestra.run.smithi046.stdout:5698 read 0x1d3e26b thru 0x1d4d3be (0xf154 bytes) 2024-09-12T07:21:04.396 INFO:teuthology.orchestra.run.smithi046.stdout:5700 writesame 0x14aeaea thru 0x14b9176 data_size 0x1(0xa68d bytes) 2024-09-12T07:21:04.397 INFO:teuthology.orchestra.run.smithi046.stdout:5701 read 0x2906686 thru 0x2909c95 (0x3610 bytes) 2024-09-12T07:21:04.397 INFO:teuthology.orchestra.run.smithi046.stdout:5702 write 0x3491cba thru 0x349db87 (0xbece bytes) 2024-09-12T07:21:04.397 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:04.397+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:04.399 INFO:teuthology.orchestra.run.smithi046.stdout:5703 write 0xd178ce8 thru 0xd17e35b (0x5674 bytes) 2024-09-12T07:21:04.402 INFO:teuthology.orchestra.run.smithi046.stdout:5704 read 0x6a8fa79 thru 0x6a9a272 (0xa7fa bytes) 2024-09-12T07:21:04.402 INFO:teuthology.orchestra.run.smithi046.stdout:5705 write 0xee8c38 thru 0xeeeeb1 (0x627a bytes) 2024-09-12T07:21:04.404 INFO:teuthology.orchestra.run.smithi046.stdout:5706 clone 44 order 23 su 4194304 sc 6 2024-09-12T07:21:04.463 INFO:teuthology.orchestra.run.smithi046.stderr:marked out osd.5. 2024-09-12T07:21:04.476 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:21:04.511 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:04.511 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:04.512 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:04.512 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:04.512 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:04.513+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:04.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:04.604 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:04.606+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:04.670 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:04.679 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:04.697 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:04.699+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:04.798 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:04.799+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:04.828 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 61.445701466999999, 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:21:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:21:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:21:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:21:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:21:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:21:04.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:21:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:21:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:21:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:21:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:21:04.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 61.382623203000001, 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.836 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:21:04.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:21:04.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:21:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 61.378918118000001, 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:21:04.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:21:04.847 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.380571838999998, 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:21:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:21:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:21:04.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116552 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5267456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.361242486999998, 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034412801999999999, 2024-09-12T07:21:04.854 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116552 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0400000000000002e-07 2024-09-12T07:21:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467850+0000", 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6529999999999998e-06 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467856+0000", 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467861+0000", 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:21:04.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502080+0000", 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034218947999999999 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502159+0000", 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9121000000000004e-05 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502260+0000", 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100846 2024-09-12T07:21:04.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116553 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5472256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.361221788000002, 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034609677999999998, 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116553 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467868+0000", 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6800000000000001e-07 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467870+0000", 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.773e-06 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467876+0000", 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0079999999999999e-06 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467880+0000", 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502304+0000", 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423924000000002 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502379+0000", 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5010000000000002e-05 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502477+0000", 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7942999999999994e-05 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116554 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5308416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.361202247999998, 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034779583000000003, 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116554 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:21:04.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467889+0000", 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7430000000000001e-06 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467895+0000", 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8159999999999999e-06 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467900+0000", 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-12T07:21:04.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502507+0000", 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034607252999999998 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502580+0000", 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3337999999999995e-05 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502667+0000", 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6468e-05 2024-09-12T07:21:04.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116555 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5349376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.361183273999998, 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034965729000000001, 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.868 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116555 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1899999999999998e-07 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467908+0000", 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5740000000000001e-06 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467914+0000", 2024-09-12T07:21:04.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9939999999999997e-06 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467918+0000", 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.622e-06 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502696+0000", 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034777798999999998 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502780+0000", 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3551e-05 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502872+0000", 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.187e-05 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116557 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3506176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:04.872 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.360088859999998, 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034335510999999999, 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116557 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469001+0000", 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469003+0000", 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469012+0000", 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8030000000000003e-06 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469019+0000", 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6959999999999999e-06 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503153+0000", 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034134096000000003 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503236+0000", 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2886000000000006e-05 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503336+0000", 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9919000000000003e-05 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116558 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3469312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.360058578999997, 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034519743999999998, 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.877 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116558 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469034+0000", 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469042+0000", 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469047+0000", 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503370+0000", 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034323201999999997 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503454+0000", 2024-09-12T07:21:04.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4109000000000004e-05 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503550+0000", 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6009999999999997e-05 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116559 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3551232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.360031747000001, 2024-09-12T07:21:04.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034689659999999997, 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116559 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469058+0000", 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469061+0000", 2024-09-12T07:21:04.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.497e-06 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469069+0000", 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2579999999999995e-06 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469074+0000", 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1279999999999999e-06 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503582+0000", 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034508351 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503656+0000", 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3234999999999999e-05 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503747+0000", 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1638999999999995e-05 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.885 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116560 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.360005018999999, 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034902912000000001, 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116560 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469085+0000", 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4700000000000001e-07 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469087+0000", 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4760000000000001e-06 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469095+0000", 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2339999999999994e-06 2024-09-12T07:21:04.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469100+0000", 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503796+0000", 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034695411000000002 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503868+0000", 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2032000000000004e-05 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503987+0000", 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119369 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116561 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3592192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:04.890 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.359981619999999, 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035102170000000002, 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116561 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469110+0000", 2024-09-12T07:21:04.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1629999999999999e-06 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469116+0000", 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.251e-06 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469121+0000", 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8210000000000001e-06 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504017+0000", 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034895772999999998 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504108+0000", 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1027999999999997e-05 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504210+0000", 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101729 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116562 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.359960776000001, 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035285642999999998, 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116562 2024-09-12T07:21:04.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469129+0000", 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469131+0000", 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1550000000000001e-06 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469138+0000", 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5520000000000001e-06 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469142+0000", 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9099999999999996e-06 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504239+0000", 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.03509669 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504311+0000", 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1650999999999996e-05 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504414+0000", 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103286 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116563 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.359940143000003, 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035471691, 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116563 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469153+0000", 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.05e-06 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469158+0000", 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.874e-06 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469163+0000", 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8069999999999999e-06 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504444+0000", 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035280793999999997 2024-09-12T07:21:04.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504523+0000", 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9036999999999996e-05 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504621+0000", 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7743000000000003e-05 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.903 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116564 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5435392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.359919032000001, 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035734657000000003, 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116564 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469171+0000", 2024-09-12T07:21:04.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469172+0000", 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.832e-06 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469179+0000", 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1449999999999996e-06 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469184+0000", 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504721+0000", 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035536890000000002 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504809+0000", 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7229000000000001e-05 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504905+0000", 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6337999999999998e-05 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116565 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5476352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.359898366000003, 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035933158, 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116565 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469193+0000", 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.973e-06 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469201+0000", 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3730000000000004e-06 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469206+0000", 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1660000000000002e-06 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504937+0000", 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035731205000000002 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505035+0000", 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7759999999999999e-05 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505124+0000", 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9258000000000005e-05 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.912 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116566 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5271552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.359875780000003, 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036116848999999999, 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116566 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469214+0000", 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:04.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469216+0000", 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8759999999999999e-06 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469222+0000", 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.173e-06 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469227+0000", 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3070000000000002e-06 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505154+0000", 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035926362000000003 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505239+0000", 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5261999999999998e-05 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505330+0000", 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1436999999999999e-05 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116567 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5312512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.359855250999999, 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036305875000000001, 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116567 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:04.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.58e-07 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469242+0000", 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469249+0000", 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5549999999999999e-06 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469254+0000", 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9899999999999997e-06 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505361+0000", 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036106923999999999 2024-09-12T07:21:04.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505449+0000", 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8531999999999999e-05 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505540+0000", 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0805999999999998e-05 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116568 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5353472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.359828577999998, 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036561383000000003, 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116568 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:04.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469263+0000", 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7549999999999999e-06 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469269+0000", 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7219999999999996e-06 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469273+0000", 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:04.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505631+0000", 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036357997000000003 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505711+0000", 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9331000000000003e-05 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505822+0000", 2024-09-12T07:21:04.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000111208 2024-09-12T07:21:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:04.926 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:04.926 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:04.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:21:04.928 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:04.929+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:04.974 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:05.056 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:05.056+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:05.076 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:05.076 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:05.076 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:05.076 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:05.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:21:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:21:05.129 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:05.131+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:05.142 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:05.142 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:05.160 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:05.160 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:05.176 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:05.177 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:05.211 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:05.228 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:05.229 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:05.231+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:05.239 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:05.239 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:05.245 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:05.245 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:05.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:21:05.262 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:05.330 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:05.332+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:05.371 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:05.371 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:05.379 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:05.464 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:05.466+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:05.528 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:05.528 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:05.533 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:05.534+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:05.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:05.634 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:05.635+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:05.684 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:05.684 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:05.694 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:05.737 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:05.739+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:05.810 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:05.854 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:05.855+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:05.931 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:05.931 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:05.936 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:05.938+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:05.940 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:06.037 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:06.038+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:06.088 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:06.088 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:06.096 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:06.138 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:06.139+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:06.239 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:06.241+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:06.252 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:06.253 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:06.262 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:06.340 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:06.340+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:06.371 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:06.371 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:06.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:06.441 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:06.442+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:06.454 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone43 from 0xd17e35c (overlap 0xaa8066) to 0x0 2024-09-12T07:21:06.513 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-12T07:21:06.538 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:06.538 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:06.547 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:06.554 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:06.556+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:06.577 INFO:teuthology.orchestra.run.smithi046.stdout:5707 punch from 0xa7e600c to 0xa7efed3, (0x9ec7 bytes) 2024-09-12T07:21:06.580 INFO:teuthology.orchestra.run.smithi046.stdout:5709 write 0x1b44d32 thru 0x1b47eeb (0x31ba bytes) 2024-09-12T07:21:06.593 INFO:teuthology.orchestra.run.smithi046.stdout:5710 read 0x42e29ce thru 0x42f0a8d (0xe0c0 bytes) 2024-09-12T07:21:06.594 INFO:teuthology.orchestra.run.smithi046.stdout:5711 read 0x3c86c79 thru 0x3c8d48a (0x6812 bytes) 2024-09-12T07:21:06.594 INFO:teuthology.orchestra.run.smithi046.stdout:5714 read 0x244ca18 thru 0x2457ca1 (0xb28a bytes) 2024-09-12T07:21:06.595 INFO:teuthology.orchestra.run.smithi046.stdout:5716 write 0x796c9b2 thru 0x79703ed (0x3a3c bytes) 2024-09-12T07:21:06.599 INFO:teuthology.orchestra.run.smithi046.stdout:5717 punch from 0xccf7f6b to 0xccfaadd, (0x2b72 bytes) 2024-09-12T07:21:06.606 INFO:teuthology.orchestra.run.smithi046.stdout:5719 writesame 0xdd1920d thru 0xdd26972 data_size 0x4f(0xd766 bytes) 2024-09-12T07:21:06.610 INFO:teuthology.orchestra.run.smithi046.stdout:5720 write 0x90653c5 thru 0x90660dc (0xd18 bytes) 2024-09-12T07:21:06.613 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:06.614 INFO:teuthology.orchestra.run.smithi046.stdout:5723 write 0xa78b030 thru 0xa78dca4 (0x2c75 bytes) 2024-09-12T07:21:06.617 INFO:teuthology.orchestra.run.smithi046.stdout:5724 write 0x786d699 thru 0x78712c2 (0x3c2a bytes) 2024-09-12T07:21:06.619 INFO:teuthology.orchestra.run.smithi046.stdout:5725 read 0xd2309b thru 0xd30443 (0xd3a9 bytes) 2024-09-12T07:21:06.619 INFO:teuthology.orchestra.run.smithi046.stdout:5726 writesame 0x6a1d351 thru 0x6a22eb2 data_size 0x15(0x5b62 bytes) 2024-09-12T07:21:06.623 INFO:teuthology.orchestra.run.smithi046.stdout:5727 read 0x3f40846 thru 0x3f49d65 (0x9520 bytes) 2024-09-12T07:21:06.624 INFO:teuthology.orchestra.run.smithi046.stdout:5728 read 0x2dca76e thru 0x2dcacab (0x53e bytes) 2024-09-12T07:21:06.624 INFO:teuthology.orchestra.run.smithi046.stdout:5729 punch from 0xa43fde2 to 0xa4429c0, (0x2bde bytes) 2024-09-12T07:21:06.630 INFO:teuthology.orchestra.run.smithi046.stdout:5731 trunc from 0xdd26973 to 0x8c3a1e7 2024-09-12T07:21:06.642 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:06.644+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:06.643 INFO:teuthology.orchestra.run.smithi046.stdout:5732 write 0x2056663 thru 0x2064cdd (0xe67b bytes) 2024-09-12T07:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout:5733 read 0x8ba96e9 thru 0x8baaa65 (0x137d bytes) 2024-09-12T07:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout:5734 read 0x6fbba2c thru 0x6fc93ff (0xd9d4 bytes) 2024-09-12T07:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout:5735 punch from 0x7ecca57 to 0x7ed6682, (0x9c2b bytes) 2024-09-12T07:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout:5737 punch from 0x64a09bb to 0x64a1b95, (0x11da bytes) 2024-09-12T07:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout:5738 trunc from 0x8c3a1e7 to 0x7c2cfdb 2024-09-12T07:21:06.665 INFO:teuthology.orchestra.run.smithi046.stdout:5740 clone 45 order 20 su 4096 sc 10 2024-09-12T07:21:06.673 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:06.674 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:06.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:06.744 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:06.745+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:06.801 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:06.801 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:06.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:06.845 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:06.846+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:06.925 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:06.925 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:06.935 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:06.945 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:06.947+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:07.047 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:07.047+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:07.052 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:07.052 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:07.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:07.146 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T07:21:07.147 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:07.148+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:07.217 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:07.217 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:07.225 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:07.266 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:07.267+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:07.274 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:07.348 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:07.350+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:07.372 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:07.372 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:07.381 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:07.399 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:07.399 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:07.415 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:07.433 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:07.433 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:07.450 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:07.450 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:07.450+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:07.450 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:07.468 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:07.468 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:07.484 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:07.485 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:07.489 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:07.489 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:07.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:07.501 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:07.518 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:07.570 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:07.572+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:07.607 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:07.607 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:07.607 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:07.607 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:07.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:07.656 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:07.658+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:07.753 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:07.755+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:07.762 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:07.762 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:07.771 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:07.854 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:07.856+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:07.919 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:07.919 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:07.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82749 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 429056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.275988705000003, 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055052166, 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.920 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82749 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644458+0000", 2024-09-12T07:21:07.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0800000000000001e-07 2024-09-12T07:21:07.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644459+0000", 2024-09-12T07:21:07.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1769999999999999e-06 2024-09-12T07:21:07.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644462+0000", 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8779999999999998e-06 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644465+0000", 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6509999999999999e-06 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699279+0000", 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054813832999999999 2024-09-12T07:21:07.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699359+0000", 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0150999999999999e-05 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699510+0000", 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015026800000000001 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82750 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 432128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.275975821000003, 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055234722, 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82750 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644470+0000", 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644471+0000", 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.17e-07 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644472+0000", 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.19e-06 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644482+0000", 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0332e-05 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644489+0000", 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7039999999999997e-06 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699549+0000", 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055060588000000001 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699592+0000", 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2707000000000003e-05 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699705+0000", 2024-09-12T07:21:07.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112984 2024-09-12T07:21:07.930 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.930 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.930 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.930 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82751 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 436224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.930 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:21:07.930 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.275948466999999, 2024-09-12T07:21:07.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055415691000000003, 2024-09-12T07:21:07.930 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.930 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82751 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644498+0000", 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644846+0000", 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034807600000000001 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644850+0000", 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1649999999999999e-06 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644860+0000", 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6949999999999995e-06 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644866+0000", 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3640000000000004e-06 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699735+0000", 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054868888999999997 2024-09-12T07:21:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699776+0000", 2024-09-12T07:21:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0722000000000001e-05 2024-09-12T07:21:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699913+0000", 2024-09-12T07:21:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013778 2024-09-12T07:21:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82752 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 439296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:21:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.275570723999998, 2024-09-12T07:21:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226917, 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82752 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:21:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644876+0000", 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644878+0000", 2024-09-12T07:21:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7850000000000001e-06 2024-09-12T07:21:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644882+0000", 2024-09-12T07:21:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9990000000000002e-06 2024-09-12T07:21:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644885+0000", 2024-09-12T07:21:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.965e-06 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699952+0000", 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067243000000002 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.699989+0000", 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7450999999999998e-05 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700102+0000", 2024-09-12T07:21:07.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113124 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82753 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 443392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.275558068000002, 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055422102000000001, 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82753 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.644888+0000", 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645016+0000", 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000127847 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645024+0000", 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7419999999999997e-06 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645030+0000", 2024-09-12T07:21:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7229999999999998e-06 2024-09-12T07:21:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645034+0000", 2024-09-12T07:21:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9380000000000002e-06 2024-09-12T07:21:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700132+0000", 2024-09-12T07:21:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055097699 2024-09-12T07:21:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700171+0000", 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9093e-05 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700310+0000", 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013905999999999999 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82754 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 446464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:21:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.275402161000002, 2024-09-12T07:21:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055424703999999998, 2024-09-12T07:21:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82754 2024-09-12T07:21:07.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.946 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:21:07.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645044+0000", 2024-09-12T07:21:07.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645318+0000", 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027397499999999998 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645322+0000", 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.766e-06 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645331+0000", 2024-09-12T07:21:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2140000000000002e-06 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645338+0000", 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0929999999999997e-06 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700330+0000", 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054991384999999997 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700369+0000", 2024-09-12T07:21:07.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:21:07.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700469+0000", 2024-09-12T07:21:07.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9913000000000004e-05 2024-09-12T07:21:07.949 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.949 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.949 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.950 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82755 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 449536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.950 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:21:07.950 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.275097170999999, 2024-09-12T07:21:07.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055388918000000002, 2024-09-12T07:21:07.950 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.950 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.950 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.950 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.950 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.950 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82755 2024-09-12T07:21:07.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.951 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:21:07.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645349+0000", 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645500+0000", 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015050999999999999 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645502+0000", 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645509+0000", 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3969999999999999e-06 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645513+0000", 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9230000000000004e-06 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700507+0000", 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054994692999999997 2024-09-12T07:21:07.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700583+0000", 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5656999999999997e-05 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700738+0000", 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154869 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82756 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 453632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.274926282999999, 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055451746000000003, 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82756 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.955 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:21:07.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645520+0000", 2024-09-12T07:21:07.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645763+0000", 2024-09-12T07:21:07.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000243105 2024-09-12T07:21:07.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645766+0000", 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0750000000000002e-06 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645774+0000", 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9370000000000004e-06 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645783+0000", 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5159999999999994e-06 2024-09-12T07:21:07.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700780+0000", 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054997329999999997 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700835+0000", 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4789e-05 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.700972+0000", 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013699400000000001 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82757 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.274651915, 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055382590000000002, 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82757 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645794+0000", 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645926+0000", 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131979 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645929+0000", 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0199999999999999e-06 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645935+0000", 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645939+0000", 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.924e-06 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701007+0000", 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055067374000000002 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701043+0000", 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6325999999999997e-05 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701177+0000", 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013387300000000001 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82758 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 460800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.274494521999998, 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055417890999999997, 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82758 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.645952+0000", 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646164+0000", 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000212189 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646167+0000", 2024-09-12T07:21:07.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-12T07:21:07.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646173+0000", 2024-09-12T07:21:07.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4139999999999999e-06 2024-09-12T07:21:07.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646177+0000", 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701208+0000", 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055031322000000001 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701254+0000", 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5701999999999997e-05 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701370+0000", 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115456 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82759 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 463872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.274261961000001, 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055376838999999997, 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.969 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82759 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646184+0000", 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646368+0000", 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183399 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646372+0000", 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9190000000000001e-06 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646380+0000", 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7819999999999999e-06 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646384+0000", 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8870000000000002e-06 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701407+0000", 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055022763000000002 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701455+0000", 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7660000000000001e-05 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701561+0000", 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010642899999999999 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.973 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82760 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.274054411999998, 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055355572999999998, 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82760 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646392+0000", 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646631+0000", 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000238798 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646634+0000", 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.016e-06 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646640+0000", 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2750000000000001e-06 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646644+0000", 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7859999999999998e-06 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701599+0000", 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054955693999999999 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701638+0000", 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8319999999999999e-05 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701747+0000", 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109684 2024-09-12T07:21:07.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82761 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.273795782999997, 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055296990999999997, 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82761 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646650+0000", 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646756+0000", 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105899 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646759+0000", 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8380000000000002e-06 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646765+0000", 2024-09-12T07:21:07.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646769+0000", 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7900000000000001e-06 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701785+0000", 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055015888999999998 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701832+0000", 2024-09-12T07:21:07.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6514000000000002e-05 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701947+0000", 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115833 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82762 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 474112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.273670215999999, 2024-09-12T07:21:07.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055346863000000003, 2024-09-12T07:21:07.983 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.983 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.983 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.983 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.983 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.983 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82762 2024-09-12T07:21:07.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.983 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.646776+0000", 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647063+0000", 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000287115 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647066+0000", 2024-09-12T07:21:07.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0189999999999998e-06 2024-09-12T07:21:07.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647075+0000", 2024-09-12T07:21:07.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7099999999999996e-06 2024-09-12T07:21:07.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647081+0000", 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6790000000000002e-06 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.701977+0000", 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054896665999999997 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702015+0000", 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.79e-05 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702123+0000", 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107774 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82763 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:21:07.987 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.273355299000002, 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055226509999999999, 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82763 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647091+0000", 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647236+0000", 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000144597 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647239+0000", 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1329999999999999e-06 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647245+0000", 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647249+0000", 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2270000000000004e-06 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702160+0000", 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054910549000000003 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702205+0000", 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5575000000000003e-05 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702317+0000", 2024-09-12T07:21:07.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011194899999999999 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82764 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 481280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.273189547999998, 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055229250000000001, 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82764 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647257+0000", 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647471+0000", 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000214625 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647474+0000", 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8509999999999999e-06 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647480+0000", 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.28e-06 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647484+0000", 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.771e-06 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702346+0000", 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054861551000000001 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702382+0000", 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.65e-05 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702486+0000", 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010367200000000001 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82765 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 485376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.272954001000002, 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055184729000000002, 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82765 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:07.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647492+0000", 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647681+0000", 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000188965 2024-09-12T07:21:07.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647693+0000", 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.225e-05 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647700+0000", 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9929999999999998e-06 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647704+0000", 2024-09-12T07:21:07.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9469999999999996e-06 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702522+0000", 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054817346000000003 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702566+0000", 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4672999999999997e-05 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702677+0000", 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110555 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82766 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 488448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.272734763000003, 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055132717999999997, 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82766 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647711+0000", 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647959+0000", 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024739599999999999 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647964+0000", 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9409999999999998e-06 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647975+0000", 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1404e-05 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647983+0000", 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2969999999999998e-06 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702706+0000", 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054723690999999998 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702743+0000", 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6464999999999997e-05 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702844+0000", 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101524 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:82767 3.17 3:eef62c18:::rbd_data.11607cf80ea.0000000000000001:d [read 492544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 59.272450722000002, 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.055054448999999998, 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82767 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.647996+0000", 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648182+0000", 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018652299999999999 2024-09-12T07:21:08.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648185+0000", 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3289999999999998e-06 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648192+0000", 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7159999999999997e-06 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.648196+0000", 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1439999999999996e-06 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702873+0000", 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054676356000000002 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.702930+0000", 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7643999999999999e-05 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.703050+0000", 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119737 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:148945 3.10 3:0cac0a45:::rbd_data.1160161ce0c8.000000000000009d:head [delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.486644761000001, 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89878390500000005, 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 148945 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433803+0000", 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.164e-06 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433807+0000", 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5000000000000001e-06 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433816+0000", 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1859999999999998e-06 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433822+0000", 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5300000000000004e-06 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433886+0000", 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4510000000000004e-05 2024-09-12T07:21:08.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433896+0000", 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0012e-05 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329359+0000", 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89546257399999996 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329585+0000", 2024-09-12T07:21:08.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022641800000000001 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329712+0000", 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126339 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.330704+0000", 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099219900000000003 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:08.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332513+0000", 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001808689 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332544+0000", 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1186999999999998e-05 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332585+0000", 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1597000000000002e-05 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.017 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.017 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.017 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:08.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:08.019 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:08.019+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:08.119 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:08.120+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:08.128 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is set 2024-09-12T07:21:08.178 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:08.178 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:08.178 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:08.179 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:08.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:08.189 INFO:teuthology.orchestra.run.smithi046.stdout:truncating image image_client.0-clone44 from 0x7c2cfdb (overlap 0x7c2cfdb) to 0x56ec4b2 2024-09-12T07:21:08.232 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:08.234+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:08.297 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:08.298 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:08.306 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:08.347 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:08.348+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:08.422 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:08.424+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:08.454 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:08.455 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:08.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:08.455 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:08.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.455 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:08.455 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:08.455 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.072098656999998, 2024-09-12T07:21:08.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:21:08.455 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:21:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:21:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:21:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:21:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:21:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:21:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.009020393, 2024-09-12T07:21:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:21:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:21:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.005315307999993, 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:21:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:21:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:21:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:21:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:21:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:21:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 61.006969028999997, 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:21:08.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.478 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116552 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5267456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.987639676999997, 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034412801999999999, 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116552 2024-09-12T07:21:08.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0400000000000002e-07 2024-09-12T07:21:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467850+0000", 2024-09-12T07:21:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6529999999999998e-06 2024-09-12T07:21:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467856+0000", 2024-09-12T07:21:08.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467861+0000", 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502080+0000", 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034218947999999999 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502159+0000", 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9121000000000004e-05 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502260+0000", 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100846 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116553 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5472256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.483 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.987618978, 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034609677999999998, 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116553 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467868+0000", 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6800000000000001e-07 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467870+0000", 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.773e-06 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467876+0000", 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0079999999999999e-06 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467880+0000", 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-12T07:21:08.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502304+0000", 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423924000000002 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502379+0000", 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5010000000000002e-05 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502477+0000", 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7942999999999994e-05 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116554 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5308416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.987599437999997, 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034779583000000003, 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116554 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:08.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467889+0000", 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7430000000000001e-06 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467895+0000", 2024-09-12T07:21:08.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8159999999999999e-06 2024-09-12T07:21:08.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467900+0000", 2024-09-12T07:21:08.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-12T07:21:08.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502507+0000", 2024-09-12T07:21:08.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034607252999999998 2024-09-12T07:21:08.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502580+0000", 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3337999999999995e-05 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502667+0000", 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6468e-05 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116555 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5349376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.987580463999997, 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034965729000000001, 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116555 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.493 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1899999999999998e-07 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467908+0000", 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5740000000000001e-06 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467914+0000", 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9939999999999997e-06 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467918+0000", 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.622e-06 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502696+0000", 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034777798999999998 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502780+0000", 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3551e-05 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502872+0000", 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.187e-05 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116557 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3506176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.986486050000003, 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034335510999999999, 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.497 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116557 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469001+0000", 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469003+0000", 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469012+0000", 2024-09-12T07:21:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8030000000000003e-06 2024-09-12T07:21:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469019+0000", 2024-09-12T07:21:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6959999999999999e-06 2024-09-12T07:21:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503153+0000", 2024-09-12T07:21:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034134096000000003 2024-09-12T07:21:08.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503236+0000", 2024-09-12T07:21:08.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2886000000000006e-05 2024-09-12T07:21:08.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503336+0000", 2024-09-12T07:21:08.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9919000000000003e-05 2024-09-12T07:21:08.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.502 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.502 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116558 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3469312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.502 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:08.502 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.986455769000003, 2024-09-12T07:21:08.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034519743999999998, 2024-09-12T07:21:08.502 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.502 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116558 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469034+0000", 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469042+0000", 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469047+0000", 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503370+0000", 2024-09-12T07:21:08.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034323201999999997 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503454+0000", 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4109000000000004e-05 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503550+0000", 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6009999999999997e-05 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116559 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3551232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.986428936999999, 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034689659999999997, 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116559 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.507 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469058+0000", 2024-09-12T07:21:08.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469061+0000", 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.497e-06 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469069+0000", 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2579999999999995e-06 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469074+0000", 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1279999999999999e-06 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503582+0000", 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034508351 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503656+0000", 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3234999999999999e-05 2024-09-12T07:21:08.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503747+0000", 2024-09-12T07:21:08.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1638999999999995e-05 2024-09-12T07:21:08.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.511 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.511 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116560 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.986402208999998, 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034902912000000001, 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116560 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469085+0000", 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4700000000000001e-07 2024-09-12T07:21:08.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469087+0000", 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4760000000000001e-06 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469095+0000", 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2339999999999994e-06 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469100+0000", 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503796+0000", 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034695411000000002 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503868+0000", 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2032000000000004e-05 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503987+0000", 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119369 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116561 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3592192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.986378809999998, 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035102170000000002, 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116561 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469110+0000", 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1629999999999999e-06 2024-09-12T07:21:08.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469116+0000", 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.251e-06 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469121+0000", 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8210000000000001e-06 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504017+0000", 2024-09-12T07:21:08.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034895772999999998 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504108+0000", 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1027999999999997e-05 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504210+0000", 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101729 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116562 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.986357966, 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035285642999999998, 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116562 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.521 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469129+0000", 2024-09-12T07:21:08.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469131+0000", 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1550000000000001e-06 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469138+0000", 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5520000000000001e-06 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469142+0000", 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9099999999999996e-06 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504239+0000", 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.03509669 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504311+0000", 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1650999999999996e-05 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504414+0000", 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103286 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116563 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.986337333000002, 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035471691, 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116563 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469153+0000", 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.05e-06 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469158+0000", 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.874e-06 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469163+0000", 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8069999999999999e-06 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504444+0000", 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035280793999999997 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504523+0000", 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9036999999999996e-05 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504621+0000", 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7743000000000003e-05 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116564 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5435392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.986316221999999, 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035734657000000003, 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116564 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469171+0000", 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:08.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469172+0000", 2024-09-12T07:21:08.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.832e-06 2024-09-12T07:21:08.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469179+0000", 2024-09-12T07:21:08.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1449999999999996e-06 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469184+0000", 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504721+0000", 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035536890000000002 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504809+0000", 2024-09-12T07:21:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7229000000000001e-05 2024-09-12T07:21:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504905+0000", 2024-09-12T07:21:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6337999999999998e-05 2024-09-12T07:21:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.534 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116565 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5476352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.986295556000002, 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035933158, 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116565 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:08.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469193+0000", 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.973e-06 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469201+0000", 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3730000000000004e-06 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469206+0000", 2024-09-12T07:21:08.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1660000000000002e-06 2024-09-12T07:21:08.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504937+0000", 2024-09-12T07:21:08.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035731205000000002 2024-09-12T07:21:08.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505035+0000", 2024-09-12T07:21:08.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7759999999999999e-05 2024-09-12T07:21:08.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505124+0000", 2024-09-12T07:21:08.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9258000000000005e-05 2024-09-12T07:21:08.540 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.540 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.541 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116566 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5271552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.541 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:08.541 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.986272970000002, 2024-09-12T07:21:08.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036116848999999999, 2024-09-12T07:21:08.541 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.541 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.541 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.541 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116566 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:08.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469214+0000", 2024-09-12T07:21:08.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:08.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469216+0000", 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8759999999999999e-06 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469222+0000", 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.173e-06 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469227+0000", 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3070000000000002e-06 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505154+0000", 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035926362000000003 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505239+0000", 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5261999999999998e-05 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505330+0000", 2024-09-12T07:21:08.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1436999999999999e-05 2024-09-12T07:21:08.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.546 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.546 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116567 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5312512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.546 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:08.546 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.986252440999998, 2024-09-12T07:21:08.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036305875000000001, 2024-09-12T07:21:08.546 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116567 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:08.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:08.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.58e-07 2024-09-12T07:21:08.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469242+0000", 2024-09-12T07:21:08.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:08.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469249+0000", 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5549999999999999e-06 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469254+0000", 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9899999999999997e-06 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505361+0000", 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036106923999999999 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505449+0000", 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8531999999999999e-05 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505540+0000", 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0805999999999998e-05 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116568 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5353472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.986225767999997, 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036561383000000003, 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116568 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.551 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469263+0000", 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7549999999999999e-06 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469269+0000", 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7219999999999996e-06 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469273+0000", 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505631+0000", 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036357997000000003 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505711+0000", 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9331000000000003e-05 2024-09-12T07:21:08.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505822+0000", 2024-09-12T07:21:08.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000111208 2024-09-12T07:21:08.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.555 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.555 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.555 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:08.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:21:08.557 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:08.558+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:08.657 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:08.659+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:08.705 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:08.705 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:08.705 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:08.705 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:08.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:21:08.759 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:08.760+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:08.821 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:08.821 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:08.830 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:21:08.870 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:08.872+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:08.961 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:08.963+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:08.979 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77440 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13156352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:21:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.734577770000001, 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051399724000000001, 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77440 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246267+0000", 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082256199999999999 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246271+0000", 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6700000000000002e-06 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246283+0000", 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.207e-05 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246291+0000", 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296645+0000", 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050353872000000001 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296729+0000", 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4215000000000006e-05 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296844+0000", 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114626 2024-09-12T07:21:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77446 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.733579254999995, 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051543643, 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77446 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:21:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246579+0000", 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013651399999999999 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246583+0000", 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1509999999999997e-06 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246593+0000", 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5670000000000001e-06 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246599+0000", 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5189999999999998e-06 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297826+0000", 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051226381000000001 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297903+0000", 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6710000000000002e-05 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297986+0000", 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3801000000000006e-05 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77447 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 5783552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.733411641000004, 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051512637, 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77447 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:21:08.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:21:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246812+0000", 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000201506 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246817+0000", 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246828+0000", 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1693e-05 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246836+0000", 2024-09-12T07:21:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4429999999999997e-06 2024-09-12T07:21:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298021+0000", 2024-09-12T07:21:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051184940999999998 2024-09-12T07:21:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298067+0000", 2024-09-12T07:21:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6292000000000003e-05 2024-09-12T07:21:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298123+0000", 2024-09-12T07:21:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5828999999999997e-05 2024-09-12T07:21:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77448 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:21:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.733171902999999, 2024-09-12T07:21:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051506024999999997, 2024-09-12T07:21:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77448 2024-09-12T07:21:08.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:21:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:21:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:08.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246892+0000", 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1854999999999998e-05 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246895+0000", 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3129999999999999e-06 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:08.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246904+0000", 2024-09-12T07:21:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6179999999999995e-06 2024-09-12T07:21:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246911+0000", 2024-09-12T07:21:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7939999999999997e-06 2024-09-12T07:21:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298164+0000", 2024-09-12T07:21:08.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051253559999999997 2024-09-12T07:21:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298249+0000", 2024-09-12T07:21:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4549000000000007e-05 2024-09-12T07:21:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298356+0000", 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107336 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77449 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.733100707000006, 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051582830000000003, 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77449 2024-09-12T07:21:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:21:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:21:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247137+0000", 2024-09-12T07:21:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021599499999999999 2024-09-12T07:21:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247142+0000", 2024-09-12T07:21:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:21:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247154+0000", 2024-09-12T07:21:09.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1878e-05 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247168+0000", 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4114000000000001e-05 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298391+0000", 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051223249999999998 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298446+0000", 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4366000000000001e-05 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298504+0000", 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8273000000000003e-05 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77450 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12681216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.732841136999994, 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051562375000000001, 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77450 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2799999999999996e-07 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247184+0000", 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247192+0000", 2024-09-12T07:21:09.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5920000000000003e-06 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247197+0000", 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6720000000000001e-06 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298550+0000", 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051352082 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298641+0000", 2024-09-12T07:21:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1236000000000005e-05 2024-09-12T07:21:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298743+0000", 2024-09-12T07:21:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102428 2024-09-12T07:21:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77451 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7061504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.732817022999996, 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051813773, 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77451 2024-09-12T07:21:09.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247335+0000", 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013012599999999999 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247340+0000", 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.544e-06 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247351+0000", 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.111e-05 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247358+0000", 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9859999999999997e-06 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298909+0000", 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051551781999999997 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298960+0000", 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0594999999999999e-05 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299019+0000", 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8629999999999999e-05 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77452 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.732652243000004, 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051892134999999999, 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77452 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247535+0000", 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165508 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247552+0000", 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6514000000000001e-05 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247565+0000", 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3372e-05 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247577+0000", 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299064+0000", 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051486697999999997 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299150+0000", 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6549000000000001e-05 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299262+0000", 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011135 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77453 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.018 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:21:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.732431521999999, 2024-09-12T07:21:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051901623000000001, 2024-09-12T07:21:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77453 2024-09-12T07:21:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:21:09.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247710+0000", 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011951 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247713+0000", 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4539999999999999e-06 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247720+0000", 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0339999999999999e-06 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247730+0000", 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0030000000000001e-05 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299297+0000", 2024-09-12T07:21:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051566446000000002 2024-09-12T07:21:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299382+0000", 2024-09-12T07:21:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4963000000000006e-05 2024-09-12T07:21:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299492+0000", 2024-09-12T07:21:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011018600000000001 2024-09-12T07:21:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77454 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7618560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:21:09.025 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.732282975000004, 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051921851999999998, 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77454 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:21:09.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247900+0000", 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016088699999999999 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247904+0000", 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6950000000000002e-06 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247912+0000", 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7129999999999994e-06 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247921+0000", 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0299999999999999e-06 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299526+0000", 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051604744000000001 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299601+0000", 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5129000000000005e-05 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299661+0000", 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9654000000000002e-05 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.029 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77455 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12746752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:21:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.732089106999993, 2024-09-12T07:21:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051971254000000001, 2024-09-12T07:21:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77455 2024-09-12T07:21:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:21:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:21:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248181+0000", 2024-09-12T07:21:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024790000000000001 2024-09-12T07:21:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248185+0000", 2024-09-12T07:21:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5079999999999999e-06 2024-09-12T07:21:09.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248197+0000", 2024-09-12T07:21:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.132e-05 2024-09-12T07:21:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248203+0000", 2024-09-12T07:21:09.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6320000000000002e-06 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299705+0000", 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051501342999999998 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299799+0000", 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4340999999999997e-05 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299904+0000", 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010521 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77456 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7979008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.731806882000001, 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051860577999999997, 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77456 2024-09-12T07:21:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:21:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:21:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248545+0000", 2024-09-12T07:21:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033009700000000001 2024-09-12T07:21:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248550+0000", 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248560+0000", 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248568+0000", 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4939999999999997e-06 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299973+0000", 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051405470000000002 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300019+0000", 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5448999999999998e-05 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300076+0000", 2024-09-12T07:21:09.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7073000000000001e-05 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77457 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13172736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.731440554000002, 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051737914000000003, 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77457 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248582+0000", 2024-09-12T07:21:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:21:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248584+0000", 2024-09-12T07:21:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:21:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248593+0000", 2024-09-12T07:21:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-12T07:21:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248609+0000", 2024-09-12T07:21:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5664000000000001e-05 2024-09-12T07:21:09.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300117+0000", 2024-09-12T07:21:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051507605999999997 2024-09-12T07:21:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300212+0000", 2024-09-12T07:21:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4919000000000004e-05 2024-09-12T07:21:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300319+0000", 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010776399999999999 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77458 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.731398135999996, 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051863578, 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.046 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77458 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248627+0000", 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1389999999999999e-06 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248635+0000", 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4019999999999996e-06 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248641+0000", 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1689999999999997e-06 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300359+0000", 2024-09-12T07:21:09.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051717889000000003 2024-09-12T07:21:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300418+0000", 2024-09-12T07:21:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8689e-05 2024-09-12T07:21:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300487+0000", 2024-09-12T07:21:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9756000000000001e-05 2024-09-12T07:21:09.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77459 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13238272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.731372442999998, 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052076504000000003, 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77459 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250611+0000", 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019617879999999999 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250618+0000", 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1829999999999997e-06 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250629+0000", 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0438000000000001e-05 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250648+0000", 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9355e-05 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300528+0000", 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049879252999999998 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300619+0000", 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1910000000000006e-05 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300726+0000", 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010657700000000001 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77467 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.055 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.729166624000001, 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05160754, 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77467 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250856+0000", 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250858+0000", 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9180000000000001e-06 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250864+0000", 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.066e-06 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250869+0000", 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302283+0000", 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051414591000000003 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302378+0000", 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4283000000000005e-05 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302463+0000", 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5249999999999999e-05 2024-09-12T07:21:09.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77468 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14909440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.729147050999998, 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051742162000000001, 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77468 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250877+0000", 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.815e-06 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250883+0000", 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250888+0000", 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302496+0000", 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051607394000000001 2024-09-12T07:21:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302553+0000", 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6971000000000001e-05 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302617+0000", 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4418999999999994e-05 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77469 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14974976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.729126871999995, 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965694, 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77469 2024-09-12T07:21:09.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250897+0000", 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9130000000000001e-06 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250903+0000", 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0129999999999999e-06 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250908+0000", 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9679999999999997e-06 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302661+0000", 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051753066 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302755+0000", 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3376999999999997e-05 2024-09-12T07:21:09.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302861+0000", 2024-09-12T07:21:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105924 2024-09-12T07:21:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77470 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12828672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:09.069 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.729107194999997, 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052098589000000001, 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77470 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:09.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250917+0000", 2024-09-12T07:21:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8190000000000001e-06 2024-09-12T07:21:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250923+0000", 2024-09-12T07:21:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:21:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250928+0000", 2024-09-12T07:21:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9420000000000003e-06 2024-09-12T07:21:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302894+0000", 2024-09-12T07:21:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965863000000001 2024-09-12T07:21:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302955+0000", 2024-09-12T07:21:09.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1235999999999994e-05 2024-09-12T07:21:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303013+0000", 2024-09-12T07:21:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8167000000000001e-05 2024-09-12T07:21:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77471 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12894208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.729087648999993, 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052326263999999997, 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77471 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.075 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256961+0000", 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060271209999999999 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256972+0000", 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0755e-05 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256986+0000", 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845e-05 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256996+0000", 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303060+0000", 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.046063661999999998 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303149+0000", 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8949999999999994e-05 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303261+0000", 2024-09-12T07:21:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011195 2024-09-12T07:21:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.079 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.079 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:09.081 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:09.081 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:09.082+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:09.182 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:09.183+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:09.235 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-12T07:21:09.237 INFO:teuthology.orchestra.run.smithi046.stdout:5741 punch from 0x2f2ea6a to 0x2f307c4, (0x1d5a bytes) 2024-09-12T07:21:09.240 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:09.240 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:09.241 INFO:teuthology.orchestra.run.smithi046.stdout:5742 punch from 0x58b2ebe to 0x58bb121, (0x8263 bytes) 2024-09-12T07:21:09.241 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:09.250 INFO:teuthology.orchestra.run.smithi046.stdout:5746 write 0xb07a493 thru 0xb08511f (0xac8d bytes) 2024-09-12T07:21:09.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:09.263 INFO:teuthology.orchestra.run.smithi046.stdout:5747 write 0xa1f6fdb thru 0xa1fa05b (0x3081 bytes) 2024-09-12T07:21:09.271 INFO:teuthology.orchestra.run.smithi046.stdout:5748 write 0x6517af5 thru 0x65242ac (0xc7b8 bytes) 2024-09-12T07:21:09.282 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:09.283+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:09.383 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:09.384+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:09.390 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:09.390 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:09.390 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:09.399 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:09.473 INFO:teuthology.orchestra.run.smithi046.stdout:5749 trunc from 0xb085120 to 0x9fd1716 2024-09-12T07:21:09.482 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 6, 7, 3] out_osds: [1, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:21:09.482 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:21:09.483 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-12T07:21:09.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-12T07:21:09.484 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:09.484+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:09.485 INFO:teuthology.orchestra.run.smithi046.stdout:5750 punch from 0x270c9e9 to 0x2719c3d, (0xd254 bytes) 2024-09-12T07:21:09.486 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:09.487 INFO:teuthology.orchestra.run.smithi046.stdout:5752 trunc from 0x9fd1716 to 0xc61d7c5 2024-09-12T07:21:09.492 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:09.493 INFO:teuthology.orchestra.run.smithi046.stdout:5754 write 0x812a75 thru 0x81aba6 (0x8132 bytes) 2024-09-12T07:21:09.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:09.585 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:09.586+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:09.604 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:09.604 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:09.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:09.604 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169456 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1527808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.084149487999999995, 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068013471000000006, 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169456 2024-09-12T07:21:09.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.606 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:09.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:09.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520228+0000", 2024-09-12T07:21:09.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.63e-07 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520230+0000", 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3420000000000002e-06 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520237+0000", 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5950000000000004e-06 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520242+0000", 2024-09-12T07:21:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588003+0000", 2024-09-12T07:21:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067761181000000004 2024-09-12T07:21:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588071+0000", 2024-09-12T07:21:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8095000000000004e-05 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588241+0000", 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016951999999999999 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169457 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1568768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.084127702999999998, 2024-09-12T07:21:09.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394231, 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169457 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:09.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520250+0000", 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6899999999999997e-07 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520252+0000", 2024-09-12T07:21:09.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1959999999999998e-06 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520259+0000", 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1509999999999998e-06 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520264+0000", 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588275+0000", 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068010854999999995 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588350+0000", 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5402999999999996e-05 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588643+0000", 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029306599999999998 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.614 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169458 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.084105227000000005, 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069257746999999995, 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169458 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:09.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:09.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520274+0000", 2024-09-12T07:21:09.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3350000000000001e-06 2024-09-12T07:21:09.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520281+0000", 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6819999999999997e-06 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520286+0000", 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0479999999999998e-06 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588740+0000", 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068453571000000005 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588810+0000", 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0611000000000005e-05 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589529+0000", 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071908599999999997 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.619 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.620 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.620 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169459 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1650688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.620 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:09.620 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.084082934999999998, 2024-09-12T07:21:09.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069503178999999998, 2024-09-12T07:21:09.620 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169459 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520296+0000", 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1890000000000001e-06 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520303+0000", 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6749999999999996e-06 2024-09-12T07:21:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520308+0000", 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1240000000000004e-06 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589563+0000", 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069254688999999994 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589637+0000", 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3711e-05 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589797+0000", 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016039799999999999 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169460 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1691648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:09.624 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.084061883000000004, 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069750706999999995, 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169460 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:09.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7399999999999999e-07 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520318+0000", 2024-09-12T07:21:09.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2639999999999998e-06 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520324+0000", 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3260000000000001e-06 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520329+0000", 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6959999999999996e-06 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589830+0000", 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069501010000000002 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589903+0000", 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3558000000000003e-05 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590066+0000", 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016247899999999999 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169461 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.084040754999999995, 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069982339000000005, 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169461 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.629 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:09.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520339+0000", 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520345+0000", 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5359999999999998e-06 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520350+0000", 2024-09-12T07:21:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.079e-06 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590098+0000", 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069747117999999997 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590168+0000", 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9982999999999997e-05 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590318+0000", 2024-09-12T07:21:09.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015076799999999999 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169462 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1773568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.084018640000000006, 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070210122, 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169462 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:09.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520359+0000", 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520361+0000", 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0109999999999999e-06 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520367+0000", 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.285e-06 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520374+0000", 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0899999999999999e-06 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590348+0000", 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069974233999999996 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590418+0000", 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9314999999999996e-05 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590568+0000", 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000150693 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169463 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.637 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.083994746999999995, 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070436171000000006, 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169463 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:09.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524036+0000", 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036543130000000002 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524043+0000", 2024-09-12T07:21:09.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0639999999999996e-06 2024-09-12T07:21:09.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524058+0000", 2024-09-12T07:21:09.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.411e-05 2024-09-12T07:21:09.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524065+0000", 2024-09-12T07:21:09.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9780000000000005e-06 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590598+0000", 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066532540000000001 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590669+0000", 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0598000000000004e-05 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590818+0000", 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014956799999999999 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169469 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.080177482999999994, 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068176724999999994, 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169469 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:09.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524200+0000", 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524202+0000", 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5100000000000001e-06 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524209+0000", 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524213+0000", 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592153+0000", 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067939948 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592225+0000", 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2078999999999999e-05 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592376+0000", 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015063000000000001 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169470 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2101248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.080156461999999998, 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394620000000003, 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169470 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524221+0000", 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8700000000000001e-07 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524223+0000", 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1619999999999998e-06 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524230+0000", 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.657e-06 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524235+0000", 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0799999999999996e-06 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592429+0000", 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068194553000000005 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592512+0000", 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3123999999999999e-05 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592615+0000", 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102657 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169471 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2142208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.080135677000000002, 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068565151000000005, 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169471 2024-09-12T07:21:09.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6300000000000001e-07 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524244+0000", 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524250+0000", 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1890000000000004e-06 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524256+0000", 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5230000000000003e-06 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592647+0000", 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068391788999999995 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592719+0000", 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1859000000000004e-05 2024-09-12T07:21:09.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592806+0000", 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6940000000000004e-05 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169472 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.080114512999999998, 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068745155000000002, 2024-09-12T07:21:09.655 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169472 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524263+0000", 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4499999999999998e-07 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524265+0000", 2024-09-12T07:21:09.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.176e-06 2024-09-12T07:21:09.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524271+0000", 2024-09-12T07:21:09.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4439999999999996e-06 2024-09-12T07:21:09.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524276+0000", 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7949999999999998e-06 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592836+0000", 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068560461000000003 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592907+0000", 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0258000000000004e-05 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593007+0000", 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100676 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169473 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2224128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.080094143000000007, 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068911932999999995, 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169473 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.34e-07 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524285+0000", 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0209999999999998e-06 2024-09-12T07:21:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524291+0000", 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8989999999999998e-06 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524296+0000", 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0069999999999997e-06 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593038+0000", 2024-09-12T07:21:09.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068742019000000001 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593109+0000", 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1075999999999995e-05 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593195+0000", 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5576999999999997e-05 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169474 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2265088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.080073949000000005, 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069079421000000002, 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169474 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3700000000000001e-07 2024-09-12T07:21:09.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524305+0000", 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1579999999999999e-06 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524311+0000", 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524316+0000", 2024-09-12T07:21:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1109999999999999e-06 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593225+0000", 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068909140999999993 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593298+0000", 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2432e-05 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593382+0000", 2024-09-12T07:21:09.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4404e-05 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169475 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2306048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.080054037999999994, 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069270256000000002, 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.669 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169475 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7500000000000001e-07 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524325+0000", 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1509999999999998e-06 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524332+0000", 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2650000000000002e-06 2024-09-12T07:21:09.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524336+0000", 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593418+0000", 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069081988999999996 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593491+0000", 2024-09-12T07:21:09.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2619000000000004e-05 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593593+0000", 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101887 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169476 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.080029957999999998, 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069449397999999996, 2024-09-12T07:21:09.673 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169476 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4900000000000001e-07 2024-09-12T07:21:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524350+0000", 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.481e-06 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524356+0000", 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6989999999999997e-06 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524362+0000", 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2320000000000001e-06 2024-09-12T07:21:09.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593623+0000", 2024-09-12T07:21:09.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069261328999999996 2024-09-12T07:21:09.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593706+0000", 2024-09-12T07:21:09.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2528999999999995e-05 2024-09-12T07:21:09.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593796+0000", 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0679000000000004e-05 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169477 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2387968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.080007152999999998, 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069660978999999998, 2024-09-12T07:21:09.678 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169477 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3299999999999998e-07 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524372+0000", 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1519999999999999e-06 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524379+0000", 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.562e-06 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524383+0000", 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5430000000000004e-06 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593852+0000", 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069468641999999997 2024-09-12T07:21:09.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593926+0000", 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4070000000000001e-05 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594031+0000", 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104677 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169478 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2428928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.079986737000000002, 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069830325999999998, 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169478 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.683 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:09.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:09.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524393+0000", 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.092e-06 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524399+0000", 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3539999999999996e-06 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524404+0000", 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9590000000000003e-06 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594062+0000", 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069658017000000003 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594131+0000", 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9226000000000004e-05 2024-09-12T07:21:09.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594220+0000", 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9295999999999993e-05 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169479 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2469888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.079966262999999996, 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069996201999999993, 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169479 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:09.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530070+0000", 2024-09-12T07:21:09.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056590099999999999 2024-09-12T07:21:09.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530075+0000", 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8200000000000002e-06 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530085+0000", 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530093+0000", 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5519999999999999e-06 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594250+0000", 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064157642000000001 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594318+0000", 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.737e-05 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594407+0000", 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9117999999999996e-05 2024-09-12T07:21:09.691 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169502 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 3411968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.073511337999999996, 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068036163999999996, 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:09.692 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169502 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:09.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535987+0000", 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051212549999999999 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535993+0000", 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0639999999999999e-06 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536003+0000", 2024-09-12T07:21:09.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0549e-05 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536012+0000", 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2309999999999996e-06 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598753+0000", 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062741665000000002 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:09.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598822+0000", 2024-09-12T07:21:09.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8689999999999995e-05 2024-09-12T07:21:09.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:09.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:09.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:09.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598902+0000", 2024-09-12T07:21:09.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9709999999999994e-05 2024-09-12T07:21:09.696 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.696 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.696 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.696 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:09.696 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:09.696 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:09.698 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:09.698 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:09.699+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:09.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-12T07:21:09.726 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:09.732 INFO:teuthology.orchestra.run.smithi046.stdout:5756 read 0x693e056 thru 0x694c6b4 (0xe65f bytes) 2024-09-12T07:21:09.733 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:09.733 INFO:teuthology.orchestra.run.smithi046.stdout:5764 write 0x2fda99 thru 0x30cbab (0xf113 bytes) 2024-09-12T07:21:09.753 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:09.766 INFO:teuthology.orchestra.run.smithi046.stdout:5765 read 0x1288389 thru 0x128b519 (0x3191 bytes) 2024-09-12T07:21:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:5767 read 0x7d5a4df thru 0x7d61270 (0x6d92 bytes) 2024-09-12T07:21:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:5768 write 0x56d55c thru 0x57a6d5 (0xd17a bytes) 2024-09-12T07:21:09.772 INFO:teuthology.orchestra.run.smithi046.stdout:5769 read 0x8e24a34 thru 0x8e320a4 (0xd671 bytes) 2024-09-12T07:21:09.773 INFO:teuthology.orchestra.run.smithi046.stdout:5770 write 0x304f558 thru 0x3058931 (0x93da bytes) 2024-09-12T07:21:09.777 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:09.777 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:09.798 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:09.799+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:09.811 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:09.821 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:09.821 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:09.821 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:09.821 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:09.829 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:09.878 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:09.878 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:09.909 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:09.913 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:09.914+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:09.943 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:09.943 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:09.952 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-12T07:21:09.982 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:09.983 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:09.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:10.032 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:10.033+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:5772 read 0x6888d9b thru 0x688ddfc (0x5062 bytes) 2024-09-12T07:21:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:5774 write 0x65f96eb thru 0x6608417 (0xed2d bytes) 2024-09-12T07:21:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:5775 writesame 0x6dc26b1 thru 0x6dce76b data_size 0x1(0xc0bb bytes) 2024-09-12T07:21:10.066 INFO:teuthology.orchestra.run.smithi046.stdout:5777 punch from 0xa8f2aaf to 0xa8f7b3e, (0x508f bytes) 2024-09-12T07:21:10.067 INFO:teuthology.orchestra.run.smithi046.stdout:5778 flatten 2024-09-12T07:21:10.102 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:10.103+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:10.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:21:10.154 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:10.154 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:10.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:10.207 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:10.208+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:10.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-12T07:21:10.284 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:10.284 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:10.293 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:10.304 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:10.305+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:10.404 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:10.405+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:10.448 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:10.448 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:10.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:10.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-12T07:21:10.537 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:10.539+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:10.606 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:10.606+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:10.614 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:10.615 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:10.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:10.709 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:10.710+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:10.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-12T07:21:10.780 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:10.780 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:10.789 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:10.807 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:10.809+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:10.908 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:10.910+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:10.953 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:10.953 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:10.962 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:11.011 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:11.012+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:11.081 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:11.081 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:11.089 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:11.110 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:11.111+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:11.163 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:21:11.211 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:11.211 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:11.211 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:11.212+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:11.220 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:11.312 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:11.314+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:11.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:11.431 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:11.433+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:11.503 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:11.503 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:11.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:11.553 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:11.554+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:11.616 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:11.617+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:11.677 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:11.677 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:11.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:11.717 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:11.718+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:11.818 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:11.819+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:11.860 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:11.860 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:11.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:11.957 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:11.958+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:11.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:12.020 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:12.021+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:12.050 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:12.050 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:12.060 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:12.120 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:12.122+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:12.123 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:12.123 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:12.153 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:12.177 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:12.191 INFO:teuthology.orchestra.run.smithi046.stdout:5780 read 0x8e20622 thru 0x8e268cf (0x62ae bytes) 2024-09-12T07:21:12.191 INFO:teuthology.orchestra.run.smithi046.stdout:5781 writesame 0xb4571c7 thru 0xb45b85a data_size 0x2(0x4694 bytes) 2024-09-12T07:21:12.195 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:12.200 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:12.201 INFO:teuthology.orchestra.run.smithi046.stdout:5783 write 0x46ab723 thru 0x46ac87a (0x1158 bytes) 2024-09-12T07:21:12.204 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:12.204 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:12.204 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:12.204 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:12.206 INFO:teuthology.orchestra.run.smithi046.stdout:5784 write 0xb1e222c thru 0xb1eda8a (0xb85f bytes) 2024-09-12T07:21:12.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:12.215 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:12.215 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:12.221 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:12.222+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:12.221 INFO:teuthology.orchestra.run.smithi046.stdout:5786 read 0x8d64230 thru 0x8d6617f (0x1f50 bytes) 2024-09-12T07:21:12.221 INFO:teuthology.orchestra.run.smithi046.stdout:5787 write 0x6e5aab9 thru 0x6e616cd (0x6c15 bytes) 2024-09-12T07:21:12.236 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:12.236 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:12.237 INFO:teuthology.orchestra.run.smithi046.stdout:5788 punch from 0x1aa8abf to 0x1ab5724, (0xcc65 bytes) 2024-09-12T07:21:12.237 INFO:teuthology.orchestra.run.smithi046.stdout:5789 write 0xa938650 thru 0xa93b39c (0x2d4d bytes) 2024-09-12T07:21:12.245 INFO:teuthology.orchestra.run.smithi046.stdout:5790 read 0x4971356 thru 0x4979a04 (0x86af bytes) 2024-09-12T07:21:12.246 INFO:teuthology.orchestra.run.smithi046.stdout:5791 punch from 0x9182d0a to 0x918bb8f, (0x8e85 bytes) 2024-09-12T07:21:12.246 INFO:teuthology.orchestra.run.smithi046.stdout:5792 read 0x696635c thru 0x696e79d (0x8442 bytes) 2024-09-12T07:21:12.247 INFO:teuthology.orchestra.run.smithi046.stdout:5793 writesame 0x6c3327 thru 0x6c87ec data_size 0x3(0x54c6 bytes) 2024-09-12T07:21:12.252 INFO:teuthology.orchestra.run.smithi046.stdout:5794 read 0xa3d2606 thru 0xa3e222c (0xfc27 bytes) 2024-09-12T07:21:12.253 INFO:teuthology.orchestra.run.smithi046.stdout:5796 punch from 0x378a5ff to 0x378d7d5, (0x31d6 bytes) 2024-09-12T07:21:12.253 INFO:teuthology.orchestra.run.smithi046.stdout:5797 read 0x391f68a thru 0x392740b (0x7d82 bytes) 2024-09-12T07:21:12.255 INFO:teuthology.orchestra.run.smithi046.stdout:5798 write 0xbbfe58a thru 0xbc0d98d (0xf404 bytes) 2024-09-12T07:21:12.257 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:12.258 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:12.273 INFO:teuthology.orchestra.run.smithi046.stdout:5800 write 0xd49baf3 thru 0xd4a4f44 (0x9452 bytes) 2024-09-12T07:21:12.277 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:12.299 INFO:teuthology.orchestra.run.smithi046.stdout:5801 writesame 0xec5b9f4 thru 0xec602ed data_size 0x1(0x48fa bytes) 2024-09-12T07:21:12.299 INFO:teuthology.orchestra.run.smithi046.stdout:5802 punch from 0x21500b4 to 0x21506cc, (0x618 bytes) 2024-09-12T07:21:12.299 INFO:teuthology.orchestra.run.smithi046.stdout:5804 read 0xe3e8a52 thru 0xe3eb669 (0x2c18 bytes) 2024-09-12T07:21:12.299 INFO:teuthology.orchestra.run.smithi046.stdout:5806 write 0xa185aa3 thru 0xa186067 (0x5c5 bytes) 2024-09-12T07:21:12.308 INFO:teuthology.orchestra.run.smithi046.stdout:5809 trunc from 0xec602ee to 0x8f72aea 2024-09-12T07:21:12.322 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:12.324+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:12.337 INFO:teuthology.orchestra.run.smithi046.stdout:5812 write 0xe31a7d8 thru 0xe31eef6 (0x471f bytes) 2024-09-12T07:21:12.338 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:12.338 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:12.346 INFO:teuthology.orchestra.run.smithi046.stdout:5813 read 0xc983924 thru 0xc990336 (0xca13 bytes) 2024-09-12T07:21:12.347 INFO:teuthology.orchestra.run.smithi046.stdout:5814 writesame 0xa55faab thru 0xa565552 data_size 0xc(0x5aa8 bytes) 2024-09-12T07:21:12.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:12.357 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:12.357 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:12.357 INFO:teuthology.orchestra.run.smithi046.stdout:5817 writesame 0xb57b71e thru 0xb588c13 data_size 0x1(0xd4f6 bytes) 2024-09-12T07:21:12.374 INFO:teuthology.orchestra.run.smithi046.stdout:5819 punch from 0xd03ef3e to 0xd03fd98, (0xe5a bytes) 2024-09-12T07:21:12.374 INFO:teuthology.orchestra.run.smithi046.stdout:5820 read 0xdc05f3b thru 0xdc09f4e (0x4014 bytes) 2024-09-12T07:21:12.374 INFO:teuthology.orchestra.run.smithi046.stdout:5821 read 0xd73a870 thru 0xd73a8d5 (0x66 bytes) 2024-09-12T07:21:12.374 INFO:teuthology.orchestra.run.smithi046.stdout:5824 writesame 0x6f84760 thru 0x6f90117 data_size 0x54(0xb9b8 bytes) 2024-09-12T07:21:12.380 INFO:teuthology.orchestra.run.smithi046.stdout:5825 write 0x17e7fc1 thru 0x17ea1d5 (0x2215 bytes) 2024-09-12T07:21:12.382 INFO:teuthology.orchestra.run.smithi046.stdout:5827 writesame 0xc9a8fa thru 0xc9e47b data_size 0x3(0x3b82 bytes) 2024-09-12T07:21:12.388 INFO:teuthology.orchestra.run.smithi046.stdout:5828 read 0x6fd4674 thru 0x6fd94bd (0x4e4a bytes) 2024-09-12T07:21:12.389 INFO:teuthology.orchestra.run.smithi046.stdout:5830 trunc from 0xe31eef7 to 0xd5856c6 2024-09-12T07:21:12.400 INFO:teuthology.orchestra.run.smithi046.stdout:5831 read 0x5201c2e thru 0x5209df3 (0x81c6 bytes) 2024-09-12T07:21:12.401 INFO:teuthology.orchestra.run.smithi046.stdout:5833 read 0x2a7de95 thru 0x2a7ecda (0xe46 bytes) 2024-09-12T07:21:12.401 INFO:teuthology.orchestra.run.smithi046.stdout:5834 write 0x4cb13ab thru 0x4cb3376 (0x1fcc bytes) 2024-09-12T07:21:12.407 INFO:teuthology.orchestra.run.smithi046.stdout:5835 read 0x6247929 thru 0x6252c3a (0xb312 bytes) 2024-09-12T07:21:12.407 INFO:teuthology.orchestra.run.smithi046.stdout:5836 writesame 0x7a74a6c thru 0x7a821cd data_size 0x13(0xd762 bytes) 2024-09-12T07:21:12.410 INFO:teuthology.orchestra.run.smithi046.stdout:5837 read 0x3da3fbe thru 0x3dae056 (0xa099 bytes) 2024-09-12T07:21:12.410 INFO:teuthology.orchestra.run.smithi046.stdout:5838 read 0x98d8096 thru 0x98dcf49 (0x4eb4 bytes) 2024-09-12T07:21:12.410 INFO:teuthology.orchestra.run.smithi046.stdout:5839 write 0xbf3c343 thru 0xbf3cfe7 (0xca5 bytes) 2024-09-12T07:21:12.418 INFO:teuthology.orchestra.run.smithi046.stdout:5840 trunc from 0xd5856c6 to 0x8a9603c 2024-09-12T07:21:12.422 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:12.424+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:12.437 INFO:teuthology.orchestra.run.smithi046.stdout:5842 trunc from 0x8a9603c to 0xe0c29dc 2024-09-12T07:21:12.442 INFO:teuthology.orchestra.run.smithi046.stdout:5844 write 0xc0b2510 thru 0xc0b83a4 (0x5e95 bytes) 2024-09-12T07:21:12.451 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:12.451 INFO:teuthology.orchestra.run.smithi046.stdout:5846 read 0x87fdf66 thru 0x880aedb (0xcf76 bytes) 2024-09-12T07:21:12.451 INFO:teuthology.orchestra.run.smithi046.stdout:5847 punch from 0x8d77a3f to 0x8d78337, (0x8f8 bytes) 2024-09-12T07:21:12.451 INFO:teuthology.orchestra.run.smithi046.stdout:5848 read 0xde3a471 thru 0xde40d8a (0x691a bytes) 2024-09-12T07:21:12.451 INFO:teuthology.orchestra.run.smithi046.stdout:5849 read 0xcbfa7e8 thru 0xcc05eb2 (0xb6cb bytes) 2024-09-12T07:21:12.463 INFO:teuthology.orchestra.run.smithi046.stdout:5850 trunc from 0xe0c29dc to 0x3789fd0 2024-09-12T07:21:12.488 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:12.488 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.105197697999998, 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:21:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:21:12.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.494 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.042119434, 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:21:12.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:12.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.500 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.038414349000007, 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:21:12.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:21:12.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:12.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:12.506 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.040068070000004, 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:12.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:21:12.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116552 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5267456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.020738717999997, 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034412801999999999, 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116552 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:12.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0400000000000002e-07 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467850+0000", 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6529999999999998e-06 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467856+0000", 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467861+0000", 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502080+0000", 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034218947999999999 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502159+0000", 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9121000000000004e-05 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502260+0000", 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100846 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116553 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5472256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.020718019, 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034609677999999998, 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116553 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467868+0000", 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6800000000000001e-07 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467870+0000", 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.773e-06 2024-09-12T07:21:12.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467876+0000", 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0079999999999999e-06 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467880+0000", 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502304+0000", 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423924000000002 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502379+0000", 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5010000000000002e-05 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502477+0000", 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7942999999999994e-05 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116554 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5308416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.020698479000004, 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034779583000000003, 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116554 2024-09-12T07:21:12.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467889+0000", 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7430000000000001e-06 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467895+0000", 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8159999999999999e-06 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467900+0000", 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502507+0000", 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034607252999999998 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502580+0000", 2024-09-12T07:21:12.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3337999999999995e-05 2024-09-12T07:21:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502667+0000", 2024-09-12T07:21:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6468e-05 2024-09-12T07:21:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.525 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116555 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5349376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.020679504999997, 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034965729000000001, 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116555 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1899999999999998e-07 2024-09-12T07:21:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467908+0000", 2024-09-12T07:21:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5740000000000001e-06 2024-09-12T07:21:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467914+0000", 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9939999999999997e-06 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467918+0000", 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.622e-06 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502696+0000", 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034777798999999998 2024-09-12T07:21:12.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502780+0000", 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3551e-05 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502872+0000", 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.187e-05 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116557 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3506176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.019585091000003, 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034335510999999999, 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116557 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.531 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469001+0000", 2024-09-12T07:21:12.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469003+0000", 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469012+0000", 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8030000000000003e-06 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469019+0000", 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6959999999999999e-06 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503153+0000", 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034134096000000003 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503236+0000", 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2886000000000006e-05 2024-09-12T07:21:12.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503336+0000", 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9919000000000003e-05 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116558 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3469312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.019554810000002, 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034519743999999998, 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116558 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469034+0000", 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:21:12.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469042+0000", 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469047+0000", 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503370+0000", 2024-09-12T07:21:12.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034323201999999997 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503454+0000", 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4109000000000004e-05 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503550+0000", 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6009999999999997e-05 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116559 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3551232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.019527977999999, 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034689659999999997, 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116559 2024-09-12T07:21:12.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469058+0000", 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469061+0000", 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.497e-06 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469069+0000", 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2579999999999995e-06 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469074+0000", 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1279999999999999e-06 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503582+0000", 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034508351 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503656+0000", 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3234999999999999e-05 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503747+0000", 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1638999999999995e-05 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116560 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.019501249999998, 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034902912000000001, 2024-09-12T07:21:12.544 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116560 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469085+0000", 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4700000000000001e-07 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469087+0000", 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4760000000000001e-06 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469095+0000", 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2339999999999994e-06 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469100+0000", 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503796+0000", 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034695411000000002 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503868+0000", 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2032000000000004e-05 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503987+0000", 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119369 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116561 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3592192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.019477850999998, 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035102170000000002, 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116561 2024-09-12T07:21:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:21:12.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469110+0000", 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1629999999999999e-06 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469116+0000", 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.251e-06 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469121+0000", 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8210000000000001e-06 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504017+0000", 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034895772999999998 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504108+0000", 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1027999999999997e-05 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504210+0000", 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101729 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116562 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.019457007, 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035285642999999998, 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.553 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116562 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469129+0000", 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469131+0000", 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1550000000000001e-06 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469138+0000", 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5520000000000001e-06 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469142+0000", 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9099999999999996e-06 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504239+0000", 2024-09-12T07:21:12.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.03509669 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504311+0000", 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1650999999999996e-05 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504414+0000", 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103286 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.557 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116563 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.019436374000001, 2024-09-12T07:21:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035471691, 2024-09-12T07:21:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.558 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116563 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469153+0000", 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.05e-06 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469158+0000", 2024-09-12T07:21:12.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.874e-06 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469163+0000", 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8069999999999999e-06 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504444+0000", 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035280793999999997 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504523+0000", 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9036999999999996e-05 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504621+0000", 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7743000000000003e-05 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116564 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5435392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.562 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.019415262999999, 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035734657000000003, 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116564 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469171+0000", 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469172+0000", 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.832e-06 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469179+0000", 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1449999999999996e-06 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469184+0000", 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:21:12.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504721+0000", 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035536890000000002 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504809+0000", 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7229000000000001e-05 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504905+0000", 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6337999999999998e-05 2024-09-12T07:21:12.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116565 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5476352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.019394597000002, 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035933158, 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116565 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469193+0000", 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.973e-06 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469201+0000", 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3730000000000004e-06 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469206+0000", 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1660000000000002e-06 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504937+0000", 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035731205000000002 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505035+0000", 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7759999999999999e-05 2024-09-12T07:21:12.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505124+0000", 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9258000000000005e-05 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116566 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5271552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.019372011000002, 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036116848999999999, 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116566 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:12.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469214+0000", 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469216+0000", 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8759999999999999e-06 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469222+0000", 2024-09-12T07:21:12.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.173e-06 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469227+0000", 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3070000000000002e-06 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505154+0000", 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035926362000000003 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505239+0000", 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5261999999999998e-05 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505330+0000", 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1436999999999999e-05 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116567 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5312512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:12.575 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.019351481999998, 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036305875000000001, 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116567 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.58e-07 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469242+0000", 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469249+0000", 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5549999999999999e-06 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469254+0000", 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9899999999999997e-06 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505361+0000", 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036106923999999999 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505449+0000", 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8531999999999999e-05 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505540+0000", 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0805999999999998e-05 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116568 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5353472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.019324808999997, 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036561383000000003, 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116568 2024-09-12T07:21:12.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:12.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469263+0000", 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7549999999999999e-06 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469269+0000", 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7219999999999996e-06 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469273+0000", 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505631+0000", 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036357997000000003 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505711+0000", 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9331000000000003e-05 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:12.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505822+0000", 2024-09-12T07:21:12.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000111208 2024-09-12T07:21:12.584 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.584 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.584 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.584 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:12.584 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:12.584 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:12.585 INFO:teuthology.orchestra.run.smithi046.stdout:5852 write 0xabb723c thru 0xabc0d69 (0x9b2e bytes) 2024-09-12T07:21:12.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:21:12.587 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:12.587 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:12.587 INFO:teuthology.orchestra.run.smithi046.stdout:5855 trunc from 0xabc0d6a to 0xa0289cf 2024-09-12T07:21:12.587 INFO:teuthology.orchestra.run.smithi046.stdout:5856 write 0xdd4e304 thru 0xdd5d845 (0xf542 bytes) 2024-09-12T07:21:12.587 INFO:teuthology.orchestra.run.smithi046.stdout:5857 write 0xe6d4e27 thru 0xe6d9cbc (0x4e96 bytes) 2024-09-12T07:21:12.587 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:12.587 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:12.587 INFO:teuthology.orchestra.run.smithi046.stdout:5863 read 0xab098d9 thru 0xab16df3 (0xd51b bytes) 2024-09-12T07:21:12.588 INFO:teuthology.orchestra.run.smithi046.stdout:5864 read 0xc1c0ebc thru 0xc1c4059 (0x319e bytes) 2024-09-12T07:21:12.588 INFO:teuthology.orchestra.run.smithi046.stdout:5866 write 0x650fc0e thru 0x651d3b0 (0xd7a3 bytes) 2024-09-12T07:21:12.588 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:12.587+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:12.588 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:12.589 INFO:teuthology.orchestra.run.smithi046.stdout:5870 write 0xb76c2df thru 0xb77bd97 (0xfab9 bytes) 2024-09-12T07:21:12.599 INFO:teuthology.orchestra.run.smithi046.stdout:5872 read 0xbd56520 thru 0xbd6577d (0xf25e bytes) 2024-09-12T07:21:12.599 INFO:teuthology.orchestra.run.smithi046.stdout:5873 writesame 0x897461 thru 0x89fd61 data_size 0x1b(0x8901 bytes) 2024-09-12T07:21:12.602 INFO:teuthology.orchestra.run.smithi046.stdout:5874 read 0xae7d7f2 thru 0xae8ceb9 (0xf6c8 bytes) 2024-09-12T07:21:12.602 INFO:teuthology.orchestra.run.smithi046.stdout:5876 write 0x929437b thru 0x9297da1 (0x3a27 bytes) 2024-09-12T07:21:12.608 INFO:teuthology.orchestra.run.smithi046.stdout:5877 punch from 0x2437dca to 0x24391f2, (0x1428 bytes) 2024-09-12T07:21:12.611 INFO:teuthology.orchestra.run.smithi046.stdout:5878 trunc from 0xe6d9cbd to 0xbf816bd 2024-09-12T07:21:12.630 INFO:teuthology.orchestra.run.smithi046.stdout:5880 write 0xd2153c6 thru 0xd21ee0e (0x9a49 bytes) 2024-09-12T07:21:12.649 INFO:teuthology.orchestra.run.smithi046.stdout:5882 writesame 0xddfe30b thru 0xddff98c data_size 0x2b(0x1682 bytes) 2024-09-12T07:21:12.649 INFO:teuthology.orchestra.run.smithi046.stdout:5883 read 0xa300344 thru 0xa3099b8 (0x9675 bytes) 2024-09-12T07:21:12.662 INFO:teuthology.orchestra.run.smithi046.stdout:5884 trunc from 0xddff98d to 0x2de37ad 2024-09-12T07:21:12.682 INFO:teuthology.orchestra.run.smithi046.stdout:5885 read 0x2ce5eab thru 0x2ce7df3 (0x1f49 bytes) 2024-09-12T07:21:12.683 INFO:teuthology.orchestra.run.smithi046.stdout:5886 punch from 0xad6f68 to 0xae52a4, (0xe33c bytes) 2024-09-12T07:21:12.683 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:12.686 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:12.688+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:12.692 INFO:teuthology.orchestra.run.smithi046.stdout:5889 writesame 0x6a25067 thru 0x6a2f497 data_size 0x1(0xa431 bytes) 2024-09-12T07:21:12.692 INFO:teuthology.orchestra.run.smithi046.stdout:5891 punch from 0x123b32d to 0x1246d6e, (0xba41 bytes) 2024-09-12T07:21:12.698 INFO:teuthology.orchestra.run.smithi046.stdout:5893 trunc from 0x6a2f498 to 0x2012a48 2024-09-12T07:21:12.715 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:12.715 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:12.715 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:12.715 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:12.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:21:12.729 INFO:teuthology.orchestra.run.smithi046.stdout:5894 write 0xd441ecd thru 0xd4447bc (0x28f0 bytes) 2024-09-12T07:21:12.734 INFO:teuthology.orchestra.run.smithi046.stdout:5898 punch from 0x5e70574 to 0x5e7935a, (0x8de6 bytes) 2024-09-12T07:21:12.747 INFO:teuthology.orchestra.run.smithi046.stdout:5899 trunc from 0xd4447bd to 0x1e27d1c 2024-09-12T07:21:12.759 INFO:teuthology.orchestra.run.smithi046.stdout:5901 punch from 0x7012da to 0x70373f, (0x2465 bytes) 2024-09-12T07:21:12.759 INFO:teuthology.orchestra.run.smithi046.stdout:5902 writesame 0x17643fb thru 0x1768253 data_size 0x1(0x3e59 bytes) 2024-09-12T07:21:12.763 INFO:teuthology.orchestra.run.smithi046.stdout:5903 trunc from 0x1e27d1c to 0x1fb63ab 2024-09-12T07:21:12.769 INFO:teuthology.orchestra.run.smithi046.stdout:5904 writesame 0x1d11a43 thru 0x1d17315 data_size 0x1(0x58d3 bytes) 2024-09-12T07:21:12.783 INFO:teuthology.orchestra.run.smithi046.stdout:5905 writesame 0x5589713 thru 0x558ffdc data_size 0x33(0x68ca bytes) 2024-09-12T07:21:12.784 INFO:teuthology.orchestra.run.smithi046.stdout:5909 read 0x1578cca thru 0x157b809 (0x2b40 bytes) 2024-09-12T07:21:12.787 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:12.788+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:12.787 INFO:teuthology.orchestra.run.smithi046.stdout:5910 trunc from 0x558ffdd to 0x84597b1 2024-09-12T07:21:12.792 INFO:teuthology.orchestra.run.smithi046.stdout:5911 write 0x3cccd67 thru 0x3cd618d (0x9427 bytes) 2024-09-12T07:21:12.804 INFO:teuthology.orchestra.run.smithi046.stdout:5912 read 0x66b6266 thru 0x66b92e7 (0x3082 bytes) 2024-09-12T07:21:12.804 INFO:teuthology.orchestra.run.smithi046.stdout:5913 write 0x7d8a92 thru 0x7da577 (0x1ae6 bytes) 2024-09-12T07:21:12.806 INFO:teuthology.orchestra.run.smithi046.stdout:5915 punch from 0x701fb0b to 0x702c36f, (0xc864 bytes) 2024-09-12T07:21:12.806 INFO:teuthology.orchestra.run.smithi046.stdout:5916 write 0x7d662b8 thru 0x7d70f9c (0xace5 bytes) 2024-09-12T07:21:12.818 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:12.818 INFO:teuthology.orchestra.run.smithi046.stdout:5918 writesame 0x7234923 thru 0x7235900 data_size 0x3(0xfde bytes) 2024-09-12T07:21:12.821 INFO:teuthology.orchestra.run.smithi046.stdout:5919 write 0x85cc43 thru 0x86c43b (0xf7f9 bytes) 2024-09-12T07:21:12.824 INFO:teuthology.orchestra.run.smithi046.stdout:5920 write 0x2f0019a thru 0x2f052ba (0x5121 bytes) 2024-09-12T07:21:12.832 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:12.842 INFO:teuthology.orchestra.run.smithi046.stdout:5923 write 0xd0e4f72 thru 0xd0e8246 (0x32d5 bytes) 2024-09-12T07:21:12.849 INFO:teuthology.orchestra.run.smithi046.stdout:5924 write 0x31b53a1 thru 0x31b600d (0xc6d bytes) 2024-09-12T07:21:12.852 INFO:teuthology.orchestra.run.smithi046.stdout:5925 read 0x9580dcf thru 0x958f8e1 (0xeb13 bytes) 2024-09-12T07:21:12.852 INFO:teuthology.orchestra.run.smithi046.stdout:5928 read 0xb97c803 thru 0xb9824e8 (0x5ce6 bytes) 2024-09-12T07:21:12.853 INFO:teuthology.orchestra.run.smithi046.stdout:5929 write 0x53eb2d1 thru 0x53f7f96 (0xccc6 bytes) 2024-09-12T07:21:12.853 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:12.853 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:12.853 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:12.853 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:12.853 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:12.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:21:12.863 INFO:teuthology.orchestra.run.smithi046.stdout:5933 write 0x8531fe3 thru 0x853fefa (0xdf18 bytes) 2024-09-12T07:21:12.876 INFO:teuthology.orchestra.run.smithi046.stdout:5934 writesame 0x2f5ec6b thru 0x2f6818f data_size 0x3(0x9525 bytes) 2024-09-12T07:21:12.885 INFO:teuthology.orchestra.run.smithi046.stdout:5935 write 0xc4ba029 thru 0xc4c0093 (0x606b bytes) 2024-09-12T07:21:12.888 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:12.889+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:12.895 INFO:teuthology.orchestra.run.smithi046.stdout:5936 read 0x881a19e thru 0x8825a13 (0xb876 bytes) 2024-09-12T07:21:12.896 INFO:teuthology.orchestra.run.smithi046.stdout:5937 punch from 0xa9bd621 to 0xa9cc53d, (0xef1c bytes) 2024-09-12T07:21:12.896 INFO:teuthology.orchestra.run.smithi046.stdout:5938 write 0x4f8a335 thru 0x4f95a30 (0xb6fc bytes) 2024-09-12T07:21:12.909 INFO:teuthology.orchestra.run.smithi046.stdout:5939 write 0x71468d5 thru 0x7155a83 (0xf1af bytes) 2024-09-12T07:21:12.918 INFO:teuthology.orchestra.run.smithi046.stdout:5940 write 0x5fccf9b thru 0x5fd1768 (0x47ce bytes) 2024-09-12T07:21:12.926 INFO:teuthology.orchestra.run.smithi046.stdout:5941 read 0xb8ea2f3 thru 0xb8f16d5 (0x73e3 bytes) 2024-09-12T07:21:12.926 INFO:teuthology.orchestra.run.smithi046.stdout:5943 write 0x4240697 thru 0x424f6bc (0xf026 bytes) 2024-09-12T07:21:12.928 INFO:teuthology.orchestra.run.smithi046.stdout:5945 read 0x5aece87 thru 0x5af9beb (0xcd65 bytes) 2024-09-12T07:21:12.929 INFO:teuthology.orchestra.run.smithi046.stdout:5948 punch from 0x1309601 to 0x130ce08, (0x3807 bytes) 2024-09-12T07:21:12.929 INFO:teuthology.orchestra.run.smithi046.stdout:5949 read 0xb916ea1 thru 0xb923b52 (0xccb2 bytes) 2024-09-12T07:21:12.930 INFO:teuthology.orchestra.run.smithi046.stdout:5952 writesame 0x6567e57 thru 0x65718f5 data_size 0x1(0x9a9f bytes) 2024-09-12T07:21:12.945 INFO:teuthology.orchestra.run.smithi046.stdout:5953 write 0x20c2814 thru 0x20c8e43 (0x6630 bytes) 2024-09-12T07:21:12.947 INFO:teuthology.orchestra.run.smithi046.stdout:5955 read 0x501cc60 thru 0x501d43a (0x7db bytes) 2024-09-12T07:21:12.948 INFO:teuthology.orchestra.run.smithi046.stdout:5957 punch from 0x9718052 to 0x9719725, (0x16d3 bytes) 2024-09-12T07:21:12.962 INFO:teuthology.orchestra.run.smithi046.stdout:5958 trunc from 0xd0e8247 to 0x1bc270 2024-09-12T07:21:12.989 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:12.990+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:13.012 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:13.012 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:13.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:13.012 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:13.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.012 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77440 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13156352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.767606381999997, 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051399724000000001, 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77440 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246267+0000", 2024-09-12T07:21:13.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082256199999999999 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246271+0000", 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6700000000000002e-06 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246283+0000", 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.207e-05 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246291+0000", 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296645+0000", 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050353872000000001 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296729+0000", 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4215000000000006e-05 2024-09-12T07:21:13.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296844+0000", 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114626 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77446 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.766607867000005, 2024-09-12T07:21:13.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051543643, 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77446 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246579+0000", 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013651399999999999 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246583+0000", 2024-09-12T07:21:13.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1509999999999997e-06 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246593+0000", 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5670000000000001e-06 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246599+0000", 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5189999999999998e-06 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297826+0000", 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051226381000000001 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297903+0000", 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6710000000000002e-05 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297986+0000", 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3801000000000006e-05 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77447 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 5783552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.766440252999999, 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051512637, 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77447 2024-09-12T07:21:13.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246812+0000", 2024-09-12T07:21:13.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000201506 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246817+0000", 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246828+0000", 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1693e-05 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246836+0000", 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4429999999999997e-06 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298021+0000", 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051184940999999998 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298067+0000", 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6292000000000003e-05 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298123+0000", 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5828999999999997e-05 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77448 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.766200514999994, 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051506024999999997, 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77448 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246892+0000", 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1854999999999998e-05 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246895+0000", 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3129999999999999e-06 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246904+0000", 2024-09-12T07:21:13.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6179999999999995e-06 2024-09-12T07:21:13.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246911+0000", 2024-09-12T07:21:13.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7939999999999997e-06 2024-09-12T07:21:13.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298164+0000", 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051253559999999997 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298249+0000", 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4549000000000007e-05 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298356+0000", 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107336 2024-09-12T07:21:13.030 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77449 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.766129319000001, 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051582830000000003, 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77449 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247137+0000", 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021599499999999999 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247142+0000", 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247154+0000", 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1878e-05 2024-09-12T07:21:13.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247168+0000", 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4114000000000001e-05 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298391+0000", 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051223249999999998 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298446+0000", 2024-09-12T07:21:13.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4366000000000001e-05 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298504+0000", 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8273000000000003e-05 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77450 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12681216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.765869749000004, 2024-09-12T07:21:13.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051562375000000001, 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77450 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2799999999999996e-07 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247184+0000", 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-12T07:21:13.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247192+0000", 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5920000000000003e-06 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247197+0000", 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6720000000000001e-06 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298550+0000", 2024-09-12T07:21:13.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051352082 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298641+0000", 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1236000000000005e-05 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298743+0000", 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102428 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77451 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7061504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.765845635000005, 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051813773, 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77451 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247335+0000", 2024-09-12T07:21:13.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013012599999999999 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247340+0000", 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.544e-06 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247351+0000", 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.111e-05 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247358+0000", 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9859999999999997e-06 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298909+0000", 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051551781999999997 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298960+0000", 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0594999999999999e-05 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299019+0000", 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8629999999999999e-05 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77452 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.765680854999999, 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051892134999999999, 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77452 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247535+0000", 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165508 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247552+0000", 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6514000000000001e-05 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247565+0000", 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3372e-05 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247577+0000", 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299064+0000", 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051486697999999997 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299150+0000", 2024-09-12T07:21:13.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6549000000000001e-05 2024-09-12T07:21:13.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299262+0000", 2024-09-12T07:21:13.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011135 2024-09-12T07:21:13.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.048 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.049 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77453 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.049 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:21:13.049 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.765460133999994, 2024-09-12T07:21:13.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051901623000000001, 2024-09-12T07:21:13.049 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77453 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247710+0000", 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011951 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247713+0000", 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4539999999999999e-06 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247720+0000", 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0339999999999999e-06 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247730+0000", 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0030000000000001e-05 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299297+0000", 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051566446000000002 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299382+0000", 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4963000000000006e-05 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299492+0000", 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011018600000000001 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77454 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7618560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.765311586999999, 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051921851999999998, 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.053 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77454 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247900+0000", 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016088699999999999 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247904+0000", 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6950000000000002e-06 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247912+0000", 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7129999999999994e-06 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247921+0000", 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0299999999999999e-06 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299526+0000", 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051604744000000001 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299601+0000", 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5129000000000005e-05 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299661+0000", 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9654000000000002e-05 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77455 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12746752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.057 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.765117719000003, 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051971254000000001, 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77455 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248181+0000", 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024790000000000001 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248185+0000", 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5079999999999999e-06 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248197+0000", 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.132e-05 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248203+0000", 2024-09-12T07:21:13.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6320000000000002e-06 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299705+0000", 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051501342999999998 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299799+0000", 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4340999999999997e-05 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299904+0000", 2024-09-12T07:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010521 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77456 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7979008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.764835493999996, 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051860577999999997, 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77456 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248545+0000", 2024-09-12T07:21:13.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033009700000000001 2024-09-12T07:21:13.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248550+0000", 2024-09-12T07:21:13.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:21:13.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248560+0000", 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248568+0000", 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4939999999999997e-06 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299973+0000", 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051405470000000002 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300019+0000", 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5448999999999998e-05 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300076+0000", 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7073000000000001e-05 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.066 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77457 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13172736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.764469165999998, 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051737914000000003, 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77457 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248582+0000", 2024-09-12T07:21:13.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:21:13.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248584+0000", 2024-09-12T07:21:13.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:21:13.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248593+0000", 2024-09-12T07:21:13.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-12T07:21:13.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248609+0000", 2024-09-12T07:21:13.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5664000000000001e-05 2024-09-12T07:21:13.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300117+0000", 2024-09-12T07:21:13.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051507605999999997 2024-09-12T07:21:13.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300212+0000", 2024-09-12T07:21:13.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4919000000000004e-05 2024-09-12T07:21:13.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300319+0000", 2024-09-12T07:21:13.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010776399999999999 2024-09-12T07:21:13.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.071 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77458 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.764426748000005, 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051863578, 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77458 2024-09-12T07:21:13.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248627+0000", 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1389999999999999e-06 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248635+0000", 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4019999999999996e-06 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248641+0000", 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1689999999999997e-06 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300359+0000", 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051717889000000003 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300418+0000", 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8689e-05 2024-09-12T07:21:13.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300487+0000", 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9756000000000001e-05 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77459 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13238272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.764401054999993, 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052076504000000003, 2024-09-12T07:21:13.076 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77459 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250611+0000", 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019617879999999999 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250618+0000", 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1829999999999997e-06 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250629+0000", 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0438000000000001e-05 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250648+0000", 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9355e-05 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300528+0000", 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049879252999999998 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300619+0000", 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1910000000000006e-05 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300726+0000", 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010657700000000001 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.080 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77467 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.762195235999997, 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05160754, 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77467 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250856+0000", 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250858+0000", 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9180000000000001e-06 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250864+0000", 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.066e-06 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250869+0000", 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302283+0000", 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051414591000000003 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302378+0000", 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4283000000000005e-05 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302463+0000", 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5249999999999999e-05 2024-09-12T07:21:13.084 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77468 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14909440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.762175662999994, 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051742162000000001, 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77468 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250877+0000", 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.815e-06 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250883+0000", 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250888+0000", 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302496+0000", 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051607394000000001 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302553+0000", 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6971000000000001e-05 2024-09-12T07:21:13.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302617+0000", 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4418999999999994e-05 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77469 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14974976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.762155484000004, 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965694, 2024-09-12T07:21:13.089 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77469 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250897+0000", 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9130000000000001e-06 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250903+0000", 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0129999999999999e-06 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250908+0000", 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9679999999999997e-06 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302661+0000", 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051753066 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302755+0000", 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3376999999999997e-05 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302861+0000", 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105924 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.093 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77470 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12828672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.762135807000007, 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052098589000000001, 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77470 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250917+0000", 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8190000000000001e-06 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250923+0000", 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250928+0000", 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9420000000000003e-06 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302894+0000", 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965863000000001 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302955+0000", 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1235999999999994e-05 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303013+0000", 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8167000000000001e-05 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.097 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77471 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12894208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.762116261000003, 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052326263999999997, 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.098 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77471 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256961+0000", 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060271209999999999 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256972+0000", 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0755e-05 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256986+0000", 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845e-05 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256996+0000", 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303060+0000", 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.046063661999999998 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303149+0000", 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8949999999999994e-05 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303261+0000", 2024-09-12T07:21:13.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011195 2024-09-12T07:21:13.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.102 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.102 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.102 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:13.102 INFO:teuthology.orchestra.run.smithi046.stdout:5959 write 0x48946d7 thru 0x48a288a (0xe1b4 bytes) 2024-09-12T07:21:13.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:13.106 INFO:teuthology.orchestra.run.smithi046.stdout:5961 read 0x3e59767 thru 0x3e62df7 (0x9691 bytes) 2024-09-12T07:21:13.106 INFO:teuthology.orchestra.run.smithi046.stdout:5962 punch from 0x1b25f25 to 0x1b2a200, (0x42db bytes) 2024-09-12T07:21:13.106 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:13.106 INFO:teuthology.orchestra.run.smithi046.stdout:5965 read 0x46c41c9 thru 0x46d2556 (0xe38e bytes) 2024-09-12T07:21:13.106 INFO:teuthology.orchestra.run.smithi046.stdout:5966 trunc from 0x48a288b to 0xf9da55 2024-09-12T07:21:13.106 INFO:teuthology.orchestra.run.smithi046.stdout:5968 read 0x9172f4 thru 0x923fb9 (0xccc6 bytes) 2024-09-12T07:21:13.106 INFO:teuthology.orchestra.run.smithi046.stdout:5969 read 0x110a6c thru 0x112ea8 (0x243d bytes) 2024-09-12T07:21:13.106 INFO:teuthology.orchestra.run.smithi046.stdout:5970 read 0x5857e0 thru 0x588dd5 (0x35f6 bytes) 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:5971 punch from 0x42ac77 to 0x439473, (0xe7fc bytes) 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:5973 punch from 0xb4fb86 to 0xb5bd45, (0xc1bf bytes) 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:5974 writesame 0x29c6611 thru 0x29d3fdf data_size 0x1(0xd9cf bytes) 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:5976 read 0xbea375 thru 0xbf049f (0x612b bytes) 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:5977 trunc from 0x29d3fe0 to 0xcea70cc 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:5978 write 0xe1e077c thru 0xe1e84f4 (0x7d79 bytes) 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:5979 read 0x6cd2def thru 0x6cd3ecb (0x10dd bytes) 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:5980 read 0x6639ee4 thru 0x663f2be (0x53db bytes) 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:5981 write 0x3533c2a thru 0x353fb1a (0xbef1 bytes) 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:5982 write 0x364d175 thru 0x3659119 (0xbfa5 bytes) 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:5986 read 0xd8fada2 thru 0xd902f8d (0x81ec bytes) 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:5987 punch from 0x172300b to 0x17233e9, (0x3de bytes) 2024-09-12T07:21:13.107 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:13.108 INFO:teuthology.orchestra.run.smithi046.stdout:5989 writesame 0xb2db6fd thru 0xb2deba0 data_size 0x6(0x34a4 bytes) 2024-09-12T07:21:13.108 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:13.107+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:13.109 INFO:teuthology.orchestra.run.smithi046.stdout:5990 punch from 0xc9f405a to 0xc9fe8e9, (0xa88f bytes) 2024-09-12T07:21:13.109 INFO:teuthology.orchestra.run.smithi046.stdout:5991 read 0xcc4fd85 thru 0xcc54ce2 (0x4f5e bytes) 2024-09-12T07:21:13.109 INFO:teuthology.orchestra.run.smithi046.stdout:5993 read 0x9996dbd thru 0x999c3a0 (0x55e4 bytes) 2024-09-12T07:21:13.109 INFO:teuthology.orchestra.run.smithi046.stdout:5994 punch from 0xa10cedb to 0xa10e3ff, (0x1524 bytes) 2024-09-12T07:21:13.109 INFO:teuthology.orchestra.run.smithi046.stdout:5995 write 0x63604f2 thru 0x6360d8f (0x89e bytes) 2024-09-12T07:21:13.109 INFO:teuthology.orchestra.run.smithi046.stdout:5996 writesame 0xad2bd thru 0xb80c0 data_size 0x94(0xae04 bytes) 2024-09-12T07:21:13.112 INFO:teuthology.orchestra.run.smithi046.stdout:5998 read 0x33e2929 thru 0x33e7cdf (0x53b7 bytes) 2024-09-12T07:21:13.113 INFO:teuthology.orchestra.run.smithi046.stdout:compare and write disabled 2024-09-12T07:21:13.113 INFO:teuthology.orchestra.run.smithi046.stdout:6000 write 0xc2b503c thru 0xc2be3e6 (0x93ab bytes) 2024-09-12T07:21:13.149 INFO:teuthology.orchestra.run.smithi046.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-12T07:21:13.181 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:21:13.206 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:13.208+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:13.229 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:13.230 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:13.230 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:13.230 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:13.239 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:13.307 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:13.309+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:13.360 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:13.360 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:13.360 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:13.360 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:13.360 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:13.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:13.409 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:13.410+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:13.509 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:13.511+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:13.536 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:13.536 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169456 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1527808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0169656959999998, 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068013471000000006, 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169456 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520228+0000", 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.63e-07 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520230+0000", 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3420000000000002e-06 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520237+0000", 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5950000000000004e-06 2024-09-12T07:21:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520242+0000", 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588003+0000", 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067761181000000004 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588071+0000", 2024-09-12T07:21:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8095000000000004e-05 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588241+0000", 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016951999999999999 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169457 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1568768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0169439110000003, 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394231, 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169457 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520250+0000", 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6899999999999997e-07 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520252+0000", 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1959999999999998e-06 2024-09-12T07:21:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520259+0000", 2024-09-12T07:21:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1509999999999998e-06 2024-09-12T07:21:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520264+0000", 2024-09-12T07:21:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588275+0000", 2024-09-12T07:21:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068010854999999995 2024-09-12T07:21:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588350+0000", 2024-09-12T07:21:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5402999999999996e-05 2024-09-12T07:21:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588643+0000", 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029306599999999998 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169458 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0169214350000004, 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069257746999999995, 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169458 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520274+0000", 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3350000000000001e-06 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520281+0000", 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6819999999999997e-06 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520286+0000", 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0479999999999998e-06 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588740+0000", 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068453571000000005 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588810+0000", 2024-09-12T07:21:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0611000000000005e-05 2024-09-12T07:21:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589529+0000", 2024-09-12T07:21:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071908599999999997 2024-09-12T07:21:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169459 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1650688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0168991429999998, 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069503178999999998, 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169459 2024-09-12T07:21:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:21:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520296+0000", 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1890000000000001e-06 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520303+0000", 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6749999999999996e-06 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520308+0000", 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1240000000000004e-06 2024-09-12T07:21:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589563+0000", 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069254688999999994 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589637+0000", 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3711e-05 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589797+0000", 2024-09-12T07:21:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016039799999999999 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169460 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1691648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0168780909999997, 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069750706999999995, 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169460 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7399999999999999e-07 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520318+0000", 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2639999999999998e-06 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520324+0000", 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3260000000000001e-06 2024-09-12T07:21:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520329+0000", 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6959999999999996e-06 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589830+0000", 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069501010000000002 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589903+0000", 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3558000000000003e-05 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590066+0000", 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016247899999999999 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169461 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0168569630000004, 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069982339000000005, 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169461 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520339+0000", 2024-09-12T07:21:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-12T07:21:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520345+0000", 2024-09-12T07:21:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5359999999999998e-06 2024-09-12T07:21:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520350+0000", 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.079e-06 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590098+0000", 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069747117999999997 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590168+0000", 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9982999999999997e-05 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590318+0000", 2024-09-12T07:21:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015076799999999999 2024-09-12T07:21:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169462 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1773568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0168348480000002, 2024-09-12T07:21:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070210122, 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169462 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520359+0000", 2024-09-12T07:21:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520361+0000", 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0109999999999999e-06 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520367+0000", 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.285e-06 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520374+0000", 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0899999999999999e-06 2024-09-12T07:21:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590348+0000", 2024-09-12T07:21:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069974233999999996 2024-09-12T07:21:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590418+0000", 2024-09-12T07:21:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9314999999999996e-05 2024-09-12T07:21:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590568+0000", 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000150693 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169463 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0168109550000004, 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070436171000000006, 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169463 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524036+0000", 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036543130000000002 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524043+0000", 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0639999999999996e-06 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524058+0000", 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.411e-05 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524065+0000", 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9780000000000005e-06 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590598+0000", 2024-09-12T07:21:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066532540000000001 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590669+0000", 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0598000000000004e-05 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590818+0000", 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014956799999999999 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169469 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0129936910000001, 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068176724999999994, 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169469 2024-09-12T07:21:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524200+0000", 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524202+0000", 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5100000000000001e-06 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524209+0000", 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524213+0000", 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592153+0000", 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067939948 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592225+0000", 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2078999999999999e-05 2024-09-12T07:21:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592376+0000", 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015063000000000001 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169470 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2101248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0129726699999999, 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394620000000003, 2024-09-12T07:21:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169470 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524221+0000", 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8700000000000001e-07 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524223+0000", 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1619999999999998e-06 2024-09-12T07:21:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524230+0000", 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.657e-06 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524235+0000", 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0799999999999996e-06 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592429+0000", 2024-09-12T07:21:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068194553000000005 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592512+0000", 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3123999999999999e-05 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592615+0000", 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102657 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169471 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2142208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0129518849999997, 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068565151000000005, 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169471 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6300000000000001e-07 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524244+0000", 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524250+0000", 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1890000000000004e-06 2024-09-12T07:21:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524256+0000", 2024-09-12T07:21:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5230000000000003e-06 2024-09-12T07:21:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592647+0000", 2024-09-12T07:21:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068391788999999995 2024-09-12T07:21:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592719+0000", 2024-09-12T07:21:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1859000000000004e-05 2024-09-12T07:21:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592806+0000", 2024-09-12T07:21:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6940000000000004e-05 2024-09-12T07:21:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169472 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.012930721, 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068745155000000002, 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169472 2024-09-12T07:21:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524263+0000", 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4499999999999998e-07 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524265+0000", 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.176e-06 2024-09-12T07:21:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524271+0000", 2024-09-12T07:21:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4439999999999996e-06 2024-09-12T07:21:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524276+0000", 2024-09-12T07:21:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7949999999999998e-06 2024-09-12T07:21:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592836+0000", 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068560461000000003 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592907+0000", 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0258000000000004e-05 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593007+0000", 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100676 2024-09-12T07:21:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169473 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2224128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0129103510000004, 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068911932999999995, 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169473 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.34e-07 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524285+0000", 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0209999999999998e-06 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524291+0000", 2024-09-12T07:21:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8989999999999998e-06 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524296+0000", 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0069999999999997e-06 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593038+0000", 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068742019000000001 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593109+0000", 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1075999999999995e-05 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593195+0000", 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5576999999999997e-05 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169474 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2265088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0128901570000002, 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069079421000000002, 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169474 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3700000000000001e-07 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524305+0000", 2024-09-12T07:21:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1579999999999999e-06 2024-09-12T07:21:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524311+0000", 2024-09-12T07:21:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-12T07:21:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524316+0000", 2024-09-12T07:21:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1109999999999999e-06 2024-09-12T07:21:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593225+0000", 2024-09-12T07:21:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068909140999999993 2024-09-12T07:21:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593298+0000", 2024-09-12T07:21:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2432e-05 2024-09-12T07:21:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593382+0000", 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4404e-05 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169475 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2306048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0128702460000003, 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069270256000000002, 2024-09-12T07:21:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169475 2024-09-12T07:21:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7500000000000001e-07 2024-09-12T07:21:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524325+0000", 2024-09-12T07:21:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1509999999999998e-06 2024-09-12T07:21:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524332+0000", 2024-09-12T07:21:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2650000000000002e-06 2024-09-12T07:21:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524336+0000", 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593418+0000", 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069081988999999996 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593491+0000", 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2619000000000004e-05 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593593+0000", 2024-09-12T07:21:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101887 2024-09-12T07:21:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169476 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0128461660000001, 2024-09-12T07:21:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069449397999999996, 2024-09-12T07:21:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169476 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4900000000000001e-07 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524350+0000", 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.481e-06 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524356+0000", 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6989999999999997e-06 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524362+0000", 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2320000000000001e-06 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593623+0000", 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069261328999999996 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593706+0000", 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2528999999999995e-05 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593796+0000", 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0679000000000004e-05 2024-09-12T07:21:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169477 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2387968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0128233609999997, 2024-09-12T07:21:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069660978999999998, 2024-09-12T07:21:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169477 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3299999999999998e-07 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524372+0000", 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1519999999999999e-06 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524379+0000", 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.562e-06 2024-09-12T07:21:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524383+0000", 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5430000000000004e-06 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593852+0000", 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069468641999999997 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593926+0000", 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4070000000000001e-05 2024-09-12T07:21:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594031+0000", 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104677 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169478 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2428928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0128029449999998, 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069830325999999998, 2024-09-12T07:21:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169478 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524393+0000", 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.092e-06 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524399+0000", 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3539999999999996e-06 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524404+0000", 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9590000000000003e-06 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594062+0000", 2024-09-12T07:21:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069658017000000003 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594131+0000", 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9226000000000004e-05 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594220+0000", 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9295999999999993e-05 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169479 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2469888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0127824710000004, 2024-09-12T07:21:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069996201999999993, 2024-09-12T07:21:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169479 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530070+0000", 2024-09-12T07:21:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056590099999999999 2024-09-12T07:21:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530075+0000", 2024-09-12T07:21:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8200000000000002e-06 2024-09-12T07:21:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530085+0000", 2024-09-12T07:21:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-12T07:21:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530093+0000", 2024-09-12T07:21:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5519999999999999e-06 2024-09-12T07:21:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594250+0000", 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064157642000000001 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594318+0000", 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.737e-05 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594407+0000", 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9117999999999996e-05 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169502 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 3411968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0063275459999996, 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068036163999999996, 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169502 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535987+0000", 2024-09-12T07:21:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051212549999999999 2024-09-12T07:21:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535993+0000", 2024-09-12T07:21:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0639999999999999e-06 2024-09-12T07:21:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536003+0000", 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0549e-05 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536012+0000", 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2309999999999996e-06 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598753+0000", 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062741665000000002 2024-09-12T07:21:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598822+0000", 2024-09-12T07:21:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8689999999999995e-05 2024-09-12T07:21:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598902+0000", 2024-09-12T07:21:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9709999999999994e-05 2024-09-12T07:21:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:13.632 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:13.632 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:13.633 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:13.634 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:13.635+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:13.734 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:13.736+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:13.741 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:13.741 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:13.741 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:13.741 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:13.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:13.835 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:13.836+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:13.896 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:13.896 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:13.897 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:13.897 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:13.897 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:13.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:13.936 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:13.938+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:14.037 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:14.038+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:14.053 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:14.053 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:14.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:14.053 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:14.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.053 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:21:14.053 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:14.053 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 76.440602471999995, 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:21:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 71.305142215999993, 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:21:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:21:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 71.300915094999993, 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:21:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:21:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 70.573164439999999, 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:21:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 70.484187851000002, 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:21:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:21:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 70.452636521000002, 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:21:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:21:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 70.449435984999994, 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:21:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:21:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 70.446190236999996, 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:21:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:21:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:21:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:21:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:21:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:21:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 70.443948406000004, 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:21:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 70.352841201999993, 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:21:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 70.349013114000002, 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:21:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:21:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89145 3.13 e53/24 3:ca7c3842:::rbd_data.11607a56846.0000000000000002:head v 53'202, mlcod=53'199)", 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 60.359825016999999, 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019835603, 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694091+0000", 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4990000000000001e-06 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697616+0000", 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035256749999999998 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697635+0000", 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8593999999999998e-05 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697646+0000", 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1436e-05 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697717+0000", 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0679000000000005e-05 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697748+0000", 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0723999999999999e-05 2024-09-12T07:21:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713886+0000", 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016138550000000002 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713924+0000", 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7446e-05 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.367100333000003, 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:21:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.355286689000003, 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:21:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:21:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 47.353950642000001, 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:21:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:21:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:21:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165942 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.8531166280000004, 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93685439599999998, 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165942 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200798+0000", 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.455e-06 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200802+0000", 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2390000000000004e-06 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200814+0000", 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1816e-05 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200820+0000", 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8760000000000002e-06 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200899+0000", 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8695999999999993e-05 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200912+0000", 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3545e-05 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137326+0000", 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93641327100000005 2024-09-12T07:21:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137519+0000", 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000193298 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137651+0000", 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013219999999999999 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165957 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.8525047509999997, 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93691399799999997, 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165957 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201409+0000", 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1999999999999999e-07 2024-09-12T07:21:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201412+0000", 2024-09-12T07:21:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2590000000000001e-06 2024-09-12T07:21:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201423+0000", 2024-09-12T07:21:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0547000000000001e-05 2024-09-12T07:21:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201428+0000", 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.101e-06 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201488+0000", 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9426999999999999e-05 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201498+0000", 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0883999999999999e-05 2024-09-12T07:21:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138057+0000", 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93655902000000002 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138168+0000", 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011034199999999999 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138322+0000", 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015469799999999999 2024-09-12T07:21:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165958 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.852473679, 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.937070242, 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165958 2024-09-12T07:21:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.58e-07 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201442+0000", 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.719e-06 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201447+0000", 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4770000000000002e-06 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201450+0000", 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.219e-06 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201530+0000", 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9833000000000006e-05 2024-09-12T07:21:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201538+0000", 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.028e-06 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138367+0000", 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93682842899999996 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138429+0000", 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1890000000000005e-05 2024-09-12T07:21:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138510+0000", 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1289e-05 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165973 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.852341054, 2024-09-12T07:21:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93711003900000001, 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165973 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201573+0000", 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8599999999999996e-07 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201580+0000", 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3810000000000004e-06 2024-09-12T07:21:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201590+0000", 2024-09-12T07:21:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0169999999999999e-05 2024-09-12T07:21:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201596+0000", 2024-09-12T07:21:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4110000000000001e-06 2024-09-12T07:21:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201645+0000", 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9156999999999997e-05 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201656+0000", 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1112e-05 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138538+0000", 2024-09-12T07:21:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93688176899999998 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138595+0000", 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6759000000000003e-05 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138682+0000", 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7293999999999995e-05 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165974 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 5.8523029060000002, 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93725339100000005, 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165974 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201611+0000", 2024-09-12T07:21:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201614+0000", 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.757e-06 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201620+0000", 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.015e-06 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201623+0000", 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6210000000000001e-06 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201679+0000", 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6017999999999998e-05 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201685+0000", 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2560000000000002e-06 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138699+0000", 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93701424600000005 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138749+0000", 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9951000000000003e-05 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138864+0000", 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115017 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:14.148 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:14.148 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:14.149 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:14.150 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:14.150+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:14.250 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:14.251+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:14.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:14.297 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:14.297 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:14.297 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:14.297 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:14.305 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:14.351 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:14.352+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:14.427 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:14.427 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:14.445 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:21:14.447 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:14.452 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:14.453+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:14.457 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:14.457 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:14.457 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:14.457 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:14.458 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:14.466 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:14.471 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:14.471 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:14.490 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:14.510 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:14.510 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:14.528 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:14.548 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:14.553 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:14.554+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:14.565 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:14.565 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:14.576 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:14.577 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:14.585 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:14.654 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:14.656+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:14.732 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:14.732 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:14.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:14.755 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:14.757+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:14.847 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:14.847 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:14.856 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:14.856 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:14.857+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:14.957 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:14.958+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:14.962 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:14.962 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:14.970 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:15.057 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:15.059+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:15.077 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:15.141 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:15.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:15.190 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:15.191+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:15.252 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:15.252 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:15.259 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:15.261+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:15.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:15.360 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:15.362+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:15.407 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:15.407 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:15.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:15.461 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:15.462+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:15.555 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:15.555 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:15.561 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:15.563+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:15.565 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:15.663 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:15.664+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:15.676 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:15.676 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:15.684 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:15.763 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:15.765+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:15.792 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:15.792 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:15.800 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:15.864 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:15.866+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:15.908 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:15.908 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:15.916 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:15.965 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:15.966+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:15.969 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 6, 7, 3] out_osds: [1, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:21:15.970 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:21:15.970 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-12T07:21:15.970 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-12T07:21:15.970 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-12T07:21:15.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-12T07:21:16.024 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:16.025 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:16.033 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:16.066 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:16.067+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:16.078 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:16.079 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "" 2024-09-12T07:21:16.079 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:16.143 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:16.143 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:16.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:21:16.167 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:16.168+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:16.259 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:16.259 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:16.268 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:21:16.268 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:16.269+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:16.369 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:16.371+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:16.416 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:16.416 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:16.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:21:16.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:21:16.509 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:16.510+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:16.571 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:16.572+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:16.576 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:16.588 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:16.588 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:16.602 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:16.686 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:16.688+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:16.709 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:16.727 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:16.727 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:16.744 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:16.744 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:16.754 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:16.754 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:16.761 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:16.761 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:16.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:16.773 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:16.774+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:16.779 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:16.797 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:16.797 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:16.821 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:16.822 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:16.839 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:16.839 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:16.874 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:16.875+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:16.887 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:16.888 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:16.888 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:16.896 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:16.981 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:16.983+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:17.047 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:17.047 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:17.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169456 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1527808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5280839249999998, 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068013471000000006, 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169456 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520228+0000", 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.63e-07 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520230+0000", 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3420000000000002e-06 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520237+0000", 2024-09-12T07:21:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5950000000000004e-06 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520242+0000", 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588003+0000", 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067761181000000004 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588071+0000", 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8095000000000004e-05 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588241+0000", 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016951999999999999 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169457 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1568768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5280621400000003, 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394231, 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169457 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520250+0000", 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6899999999999997e-07 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520252+0000", 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1959999999999998e-06 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520259+0000", 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1509999999999998e-06 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520264+0000", 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588275+0000", 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068010854999999995 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588350+0000", 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5402999999999996e-05 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588643+0000", 2024-09-12T07:21:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029306599999999998 2024-09-12T07:21:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169458 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5280396639999996, 2024-09-12T07:21:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069257746999999995, 2024-09-12T07:21:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169458 2024-09-12T07:21:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520274+0000", 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3350000000000001e-06 2024-09-12T07:21:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520281+0000", 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6819999999999997e-06 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520286+0000", 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0479999999999998e-06 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588740+0000", 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068453571000000005 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588810+0000", 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0611000000000005e-05 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589529+0000", 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071908599999999997 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169459 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1650688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5280173719999999, 2024-09-12T07:21:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069503178999999998, 2024-09-12T07:21:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169459 2024-09-12T07:21:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:17.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:17.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:21:17.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520296+0000", 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1890000000000001e-06 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520303+0000", 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6749999999999996e-06 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520308+0000", 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1240000000000004e-06 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589563+0000", 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069254688999999994 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589637+0000", 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3711e-05 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589797+0000", 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016039799999999999 2024-09-12T07:21:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169460 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1691648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5279963199999997, 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069750706999999995, 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169460 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7399999999999999e-07 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520318+0000", 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2639999999999998e-06 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520324+0000", 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3260000000000001e-06 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520329+0000", 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6959999999999996e-06 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589830+0000", 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069501010000000002 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589903+0000", 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3558000000000003e-05 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590066+0000", 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016247899999999999 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169461 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5279751919999995, 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069982339000000005, 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169461 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520339+0000", 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520345+0000", 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5359999999999998e-06 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520350+0000", 2024-09-12T07:21:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.079e-06 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590098+0000", 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069747117999999997 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590168+0000", 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9982999999999997e-05 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590318+0000", 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015076799999999999 2024-09-12T07:21:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169462 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1773568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5279530770000003, 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070210122, 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169462 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520359+0000", 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520361+0000", 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0109999999999999e-06 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520367+0000", 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.285e-06 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520374+0000", 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0899999999999999e-06 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590348+0000", 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069974233999999996 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590418+0000", 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9314999999999996e-05 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590568+0000", 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000150693 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169463 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5279291839999996, 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070436171000000006, 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169463 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524036+0000", 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036543130000000002 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524043+0000", 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0639999999999996e-06 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524058+0000", 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.411e-05 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524065+0000", 2024-09-12T07:21:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9780000000000005e-06 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590598+0000", 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066532540000000001 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590669+0000", 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0598000000000004e-05 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590818+0000", 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014956799999999999 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169469 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5241119200000002, 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068176724999999994, 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169469 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524200+0000", 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524202+0000", 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5100000000000001e-06 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524209+0000", 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524213+0000", 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592153+0000", 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067939948 2024-09-12T07:21:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592225+0000", 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2078999999999999e-05 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592376+0000", 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015063000000000001 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169470 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2101248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5240908989999999, 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394620000000003, 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169470 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524221+0000", 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8700000000000001e-07 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524223+0000", 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1619999999999998e-06 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524230+0000", 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.657e-06 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524235+0000", 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0799999999999996e-06 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592429+0000", 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068194553000000005 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592512+0000", 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3123999999999999e-05 2024-09-12T07:21:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592615+0000", 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102657 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169471 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2142208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5240701139999997, 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068565151000000005, 2024-09-12T07:21:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169471 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6300000000000001e-07 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524244+0000", 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524250+0000", 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1890000000000004e-06 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524256+0000", 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5230000000000003e-06 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592647+0000", 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068391788999999995 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592719+0000", 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1859000000000004e-05 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592806+0000", 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6940000000000004e-05 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169472 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5240489500000001, 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068745155000000002, 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169472 2024-09-12T07:21:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524263+0000", 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4499999999999998e-07 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524265+0000", 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.176e-06 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524271+0000", 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4439999999999996e-06 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524276+0000", 2024-09-12T07:21:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7949999999999998e-06 2024-09-12T07:21:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592836+0000", 2024-09-12T07:21:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068560461000000003 2024-09-12T07:21:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592907+0000", 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0258000000000004e-05 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593007+0000", 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100676 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169473 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2224128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5240285799999995, 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068911932999999995, 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169473 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.34e-07 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524285+0000", 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0209999999999998e-06 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524291+0000", 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8989999999999998e-06 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524296+0000", 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0069999999999997e-06 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593038+0000", 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068742019000000001 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593109+0000", 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1075999999999995e-05 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593195+0000", 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5576999999999997e-05 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169474 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2265088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5240083860000002, 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069079421000000002, 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169474 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3700000000000001e-07 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524305+0000", 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1579999999999999e-06 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524311+0000", 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524316+0000", 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1109999999999999e-06 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593225+0000", 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068909140999999993 2024-09-12T07:21:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593298+0000", 2024-09-12T07:21:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2432e-05 2024-09-12T07:21:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593382+0000", 2024-09-12T07:21:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4404e-05 2024-09-12T07:21:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169475 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2306048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5239884750000003, 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069270256000000002, 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169475 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7500000000000001e-07 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524325+0000", 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1509999999999998e-06 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524332+0000", 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2650000000000002e-06 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524336+0000", 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593418+0000", 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069081988999999996 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593491+0000", 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2619000000000004e-05 2024-09-12T07:21:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593593+0000", 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101887 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169476 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5239643950000001, 2024-09-12T07:21:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069449397999999996, 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169476 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4900000000000001e-07 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524350+0000", 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.481e-06 2024-09-12T07:21:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524356+0000", 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6989999999999997e-06 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524362+0000", 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2320000000000001e-06 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593623+0000", 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069261328999999996 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593706+0000", 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2528999999999995e-05 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593796+0000", 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0679000000000004e-05 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169477 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2387968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5239415899999997, 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069660978999999998, 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169477 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3299999999999998e-07 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524372+0000", 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1519999999999999e-06 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524379+0000", 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.562e-06 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524383+0000", 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5430000000000004e-06 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593852+0000", 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069468641999999997 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593926+0000", 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4070000000000001e-05 2024-09-12T07:21:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594031+0000", 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104677 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169478 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2428928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5239211739999998, 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069830325999999998, 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169478 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524393+0000", 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.092e-06 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524399+0000", 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3539999999999996e-06 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524404+0000", 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9590000000000003e-06 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594062+0000", 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069658017000000003 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594131+0000", 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9226000000000004e-05 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594220+0000", 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9295999999999993e-05 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169479 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2469888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5239007000000004, 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069996201999999993, 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169479 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530070+0000", 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056590099999999999 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530075+0000", 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8200000000000002e-06 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530085+0000", 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530093+0000", 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5519999999999999e-06 2024-09-12T07:21:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594250+0000", 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064157642000000001 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594318+0000", 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.737e-05 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594407+0000", 2024-09-12T07:21:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9117999999999996e-05 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169502 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 3411968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.5174457749999997, 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068036163999999996, 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169502 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535987+0000", 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051212549999999999 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535993+0000", 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0639999999999999e-06 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536003+0000", 2024-09-12T07:21:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0549e-05 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536012+0000", 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2309999999999996e-06 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598753+0000", 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062741665000000002 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598822+0000", 2024-09-12T07:21:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8689999999999995e-05 2024-09-12T07:21:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598902+0000", 2024-09-12T07:21:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9709999999999994e-05 2024-09-12T07:21:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:17.139 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:17.141 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:17.141 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:17.142+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:17.242 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:17.243+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:17.260 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:17.260 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:17.269 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:17.342 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:17.344+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:17.376 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:17.377 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:17.377 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:17.377 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:17.377 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:17.385 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:17.458 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:21:17.469 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:17.471+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:17.541 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:17.541 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:17.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:17.541 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:17.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.928517846000005, 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:21:17.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 74.793057590000004, 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.546 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.547 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:17.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:21:17.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:21:17.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:21:17.551 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.551 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.551 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.551 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:21:17.551 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:17.551 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 74.788830469000004, 2024-09-12T07:21:17.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:21:17.551 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:21:17.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:21:17.555 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 74.061079813999996, 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:21:17.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:21:17.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:21:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:17.561 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 73.972103224999998, 2024-09-12T07:21:17.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:21:17.561 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.561 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.561 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.561 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:17.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:17.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:21:17.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:21:17.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:21:17.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:17.565 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 73.940551894999999, 2024-09-12T07:21:17.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:21:17.566 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.566 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.566 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.566 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:17.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:21:17.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:21:17.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 73.937351359000004, 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:17.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:17.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:21:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:21:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:21:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:21:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:21:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:21:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:21:17.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 73.934105611000007, 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:21:17.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 73.93186378, 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.579 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:21:17.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:21:17.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:21:17.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:21:17.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.583 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 73.840756576000004, 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:21:17.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:21:17.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:21:17.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:21:17.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:21:17.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:21:17.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:21:17.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:21:17.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.588 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 73.836928487999998, 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:17.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:21:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:21:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:21:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:21:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:21:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:21:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:21:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:21:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:21:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:89145 3.13 e53/24 3:ca7c3842:::rbd_data.11607a56846.0000000000000002:head v 53'202, mlcod=53'199)", 2024-09-12T07:21:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:21:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 63.847740391000002, 2024-09-12T07:21:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019835603, 2024-09-12T07:21:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694088+0000", 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.694091+0000", 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4990000000000001e-06 2024-09-12T07:21:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697616+0000", 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0035256749999999998 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697635+0000", 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8593999999999998e-05 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697646+0000", 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1436e-05 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697717+0000", 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0679000000000005e-05 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.697748+0000", 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0723999999999999e-05 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713886+0000", 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016138550000000002 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:13.713924+0000", 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7446e-05 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:21:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 50.855015707, 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:21:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:21:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:21:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:21:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.601 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:21:17.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:21:17.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:21:17.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:21:17.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:21:17.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:21:17.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:21:17.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:21:17.603 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.603 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.603 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.603 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:21:17.603 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:17.603 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 50.843202063, 2024-09-12T07:21:17.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:21:17.604 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.604 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.604 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:17.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:21:17.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:21:17.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:21:17.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:21:17.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:21:17.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:21:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:21:17.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:21:17.608 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.608 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.608 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.608 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:21:17.608 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:17.608 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 50.841866015999997, 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:17.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:21:17.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:21:17.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:21:17.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:21:17.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:21:17.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:21:17.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:21:17.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:21:17.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:17.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:21:17.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:21:17.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165942 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.3410320020000004, 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93685439599999998, 2024-09-12T07:21:17.613 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165942 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200798+0000", 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.455e-06 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200802+0000", 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2390000000000004e-06 2024-09-12T07:21:17.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200814+0000", 2024-09-12T07:21:17.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1816e-05 2024-09-12T07:21:17.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200820+0000", 2024-09-12T07:21:17.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8760000000000002e-06 2024-09-12T07:21:17.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200899+0000", 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8695999999999993e-05 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200912+0000", 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3545e-05 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137326+0000", 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93641327100000005 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137519+0000", 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000193298 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137651+0000", 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013219999999999999 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165957 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.3404201249999996, 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93691399799999997, 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165957 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201409+0000", 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1999999999999999e-07 2024-09-12T07:21:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201412+0000", 2024-09-12T07:21:17.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2590000000000001e-06 2024-09-12T07:21:17.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201423+0000", 2024-09-12T07:21:17.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0547000000000001e-05 2024-09-12T07:21:17.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201428+0000", 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.101e-06 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201488+0000", 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9426999999999999e-05 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:17.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201498+0000", 2024-09-12T07:21:17.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0883999999999999e-05 2024-09-12T07:21:17.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138057+0000", 2024-09-12T07:21:17.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93655902000000002 2024-09-12T07:21:17.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138168+0000", 2024-09-12T07:21:17.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011034199999999999 2024-09-12T07:21:17.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138322+0000", 2024-09-12T07:21:17.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015469799999999999 2024-09-12T07:21:17.624 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.624 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.624 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.624 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165958 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:17.624 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:17.624 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.3403890529999991, 2024-09-12T07:21:17.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.937070242, 2024-09-12T07:21:17.625 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.625 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:17.625 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:17.625 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:17.625 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.625 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165958 2024-09-12T07:21:17.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.625 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:17.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.58e-07 2024-09-12T07:21:17.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201442+0000", 2024-09-12T07:21:17.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.719e-06 2024-09-12T07:21:17.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201447+0000", 2024-09-12T07:21:17.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4770000000000002e-06 2024-09-12T07:21:17.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201450+0000", 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.219e-06 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201530+0000", 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9833000000000006e-05 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:17.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201538+0000", 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.028e-06 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138367+0000", 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93682842899999996 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138429+0000", 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1890000000000005e-05 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138510+0000", 2024-09-12T07:21:17.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1289e-05 2024-09-12T07:21:17.630 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.630 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.630 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.630 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165973 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:17.630 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.340256428, 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93711003900000001, 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165973 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:17.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:17.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201573+0000", 2024-09-12T07:21:17.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8599999999999996e-07 2024-09-12T07:21:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201580+0000", 2024-09-12T07:21:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3810000000000004e-06 2024-09-12T07:21:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201590+0000", 2024-09-12T07:21:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0169999999999999e-05 2024-09-12T07:21:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201596+0000", 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4110000000000001e-06 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201645+0000", 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9156999999999997e-05 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201656+0000", 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1112e-05 2024-09-12T07:21:17.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138538+0000", 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93688176899999998 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138595+0000", 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6759000000000003e-05 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138682+0000", 2024-09-12T07:21:17.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7293999999999995e-05 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165974 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.3402182800000002, 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93725339100000005, 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:17.636 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165974 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:17.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201611+0000", 2024-09-12T07:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201614+0000", 2024-09-12T07:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.757e-06 2024-09-12T07:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201620+0000", 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.015e-06 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201623+0000", 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6210000000000001e-06 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201679+0000", 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6017999999999998e-05 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:17.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201685+0000", 2024-09-12T07:21:17.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2560000000000002e-06 2024-09-12T07:21:17.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:17.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138699+0000", 2024-09-12T07:21:17.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93701424600000005 2024-09-12T07:21:17.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:17.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138749+0000", 2024-09-12T07:21:17.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9951000000000003e-05 2024-09-12T07:21:17.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:17.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:17.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:17.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138864+0000", 2024-09-12T07:21:17.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115017 2024-09-12T07:21:17.641 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.641 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.641 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.641 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:17.641 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:17.641 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:17.643 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:17.643 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:17.644+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:17.744 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:17.745+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:17.791 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:17.791 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:17.791 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:17.791 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:17.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:17.844 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:17.846+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:17.906 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:17.906 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:17.906 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:17.906 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:17.907 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:17.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:17.945 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:17.947+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:18.023 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:18.023 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:18.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:18.023 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.542634164000006, 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:21:18.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:21:18.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.542607283999999, 2024-09-12T07:21:18.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:21:18.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:21:18.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:21:18.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:21:18.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:21:18.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:21:18.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:21:18.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.542583264000001, 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.033 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:18.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.037 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.542555617999994, 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:18.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:21:18.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:21:18.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:21:18.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.042 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.542532155999993, 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:21:18.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:21:18.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.542509628999994, 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.047 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.051 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.542487825999999, 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:21:18.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.535854045999997, 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.056 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.057 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.057 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:21:18.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.057 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:18.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:21:18.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.060 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.061 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.061 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.061 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:18.061 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.535834038000004, 2024-09-12T07:21:18.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:21:18.061 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.061 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.061 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.061 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:21:18.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.529351005999999, 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.066 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:18.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:18.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:21:18.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.529315718999996, 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:18.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:21:18.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.529293085999996, 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:21:18.075 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:21:18.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:21:18.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.529239031000003, 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:21:18.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:21:18.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:18.084 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.529220089999995, 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:18.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:21:18.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.529208152999999, 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.089 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:18.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.093 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.529196537999994, 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:21:18.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:21:18.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:21:18.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:21:18.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:21:18.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:21:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:21:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:21:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:21:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:21:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:18.098 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.522183929999997, 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.102 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.522160215, 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.103 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.107 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.522029357999998, 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:21:18.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.111 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 74.558358568000003, 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.112 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:21:18.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:21:18.117 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.118 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.118 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.118 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.118 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.118 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:18.119 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:18.120 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:18.120+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:18.220 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:18.222+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:18.266 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:18.267 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:18.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:18.321 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:18.322+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:18.385 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:18.385 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:18.385 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:18.385 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:18.385 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:18.391 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:18.434 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:18.436+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:18.503 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:21:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:18.503 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 85.974239112999996, 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:21:18.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.508 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.509 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.509 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.509 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:18.509 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.122441447, 2024-09-12T07:21:18.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:21:18.509 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.509 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.509 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.509 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.509 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:18.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.12231817, 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:21:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:21:18.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.122114779, 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.518 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:21:18.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.522 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.121978952000006, 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:18.523 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:21:18.524 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:21:18.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.526 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.121652295999993, 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.528 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:21:18.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.530 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.121613408000002, 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.531 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.535 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.535 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:21:18.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:21:18.535 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.535 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.535 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.535 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.535 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:18.535 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.121246533999994, 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.536 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.537 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.539 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.121111678000005, 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:21:18.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.541 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.120741787, 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.544 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:21:18.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.120551589000002, 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:21:18.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.550 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:18.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:18.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:21:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.12033151, 2024-09-12T07:21:18.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:21:18.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:21:18.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.120187845000004, 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:21:18.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:21:18.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.119961673000006, 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.562 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.566 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.119761315000005, 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:21:18.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:21:18.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:21:18.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:21:18.571 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.571 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.571 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.571 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.571 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:18.571 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.119353431999997, 2024-09-12T07:21:18.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:21:18.571 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:21:18.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.575 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.118887549999997, 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.576 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:21:18.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.117285339999995, 2024-09-12T07:21:18.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.116751304999994, 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.585 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:21:18.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:18.589 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.860553234999998, 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:18.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:21:18.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:21:18.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:21:18.594 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.594 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.594 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.594 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.594 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.594 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:18.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:18.596 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:18.597+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:18.696 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:18.698+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:18.743 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:18.743 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:18.743 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:18.743 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:18.751 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:18.797 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:18.799+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:18.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:18.859 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:18.859 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:18.859 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:18.859 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:18.859 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:18.867 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:18.910 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:18.912+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:18.975 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:18.976 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:18.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:18.976 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:18.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.976 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:18.976 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:18.976 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.427147618000006, 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:21:18.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:21:18.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:21:18.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:21:18.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:21:18.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:21:18.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:21:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:21:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:18.981 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.427127080999995, 2024-09-12T07:21:18.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:21:18.982 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.982 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.982 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.982 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.982 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.982 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.982 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:21:18.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.982 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:18.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:18.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:18.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.427107078000006, 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.987 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:18.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:18.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:21:18.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:21:18.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:21:18.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:21:18.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:18.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.427087129, 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:21:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.993 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.993 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.993 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.993 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:18.993 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:21:18.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.993 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:18.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:18.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:18.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:18.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:18.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:18.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:18.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:18.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:21:18.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:18.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:21:18.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:21:18.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:18.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:21:18.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:21:18.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:21:18.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:18.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:21:18.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:21:18.997 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.997 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:18.997 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:18.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:18.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:18.997 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:18.998 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:18.998 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.427066260999993, 2024-09-12T07:21:18.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:21:18.998 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:18.998 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:18.998 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:18.998 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:18.998 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:18.998 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.000 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:21:19.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.000 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:19.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:19.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:19.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:21:19.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:21:19.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:21:19.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:21:19.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:21:19.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:21:19.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:21:19.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:21:19.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:21:19.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:21:19.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:21:19.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:21:19.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:21:19.006 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.006 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.006 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.006 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.006 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:19.006 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.427044559999999, 2024-09-12T07:21:19.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:21:19.007 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.007 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.007 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.007 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.008 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.008 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.008 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:21:19.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.008 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:19.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:19.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:21:19.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:21:19.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:21:19.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:21:19.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:21:19.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:21:19.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:21:19.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:21:19.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:21:19.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:21:19.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:21:19.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:21:19.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:21:19.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:21:19.015 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.015 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.015 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.015 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.015 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:19.015 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.427024119999999, 2024-09-12T07:21:19.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:21:19.016 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.016 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.016 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.016 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.016 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.016 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:21:19.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.016 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:21:19.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:21:19.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:21:19.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:21:19.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:21:19.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:21:19.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:21:19.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:21:19.020 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.020 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.020 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.426996399000004, 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.021 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:19.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:19.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:21:19.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:21:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:21:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:21:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:21:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:21:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:21:19.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:21:19.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:21:19.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:21:19.025 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.025 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.026 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.026 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.026 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:19.026 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.426973464, 2024-09-12T07:21:19.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:21:19.026 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:19.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:21:19.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:21:19.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:21:19.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:21:19.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:21:19.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:21:19.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.426953163999997, 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.031 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:21:19.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.032 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:19.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:19.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:19.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:19.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:21:19.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:21:19.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:21:19.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:21:19.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:21:19.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.426932887999996, 2024-09-12T07:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:21:19.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:21:19.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:21:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:21:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.426912728999994, 2024-09-12T07:21:19.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:21:19.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:21:19.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.426775531000004, 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.046 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:19.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:21:19.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:21:19.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:21:19.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:21:19.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:21:19.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:21:19.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:21:19.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:21:19.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:21:19.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:21:19.050 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.426752883000006, 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.051 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.052 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.052 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:21:19.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.052 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:19.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:21:19.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:21:19.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:21:19.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:21:19.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:21:19.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:21:19.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:21:19.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:21:19.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:21:19.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:21:19.056 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.056 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.056 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.056 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68088 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 393216~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.056 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:19.056 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.420310123999997, 2024-09-12T07:21:19.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10346986399999999, 2024-09-12T07:21:19.056 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.056 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68088 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555923+0000", 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.891e-06 2024-09-12T07:21:19.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555929+0000", 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0549999999999996e-06 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555934+0000", 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6419999999999998e-06 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659060+0000", 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103125862 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659139+0000", 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9450000000000007e-05 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659391+0000", 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025158400000000001 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.060 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.061 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.061 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.061 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:19.061 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.420290897000001, 2024-09-12T07:21:19.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:21:19.061 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.061 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.061 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:21:19.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:21:19.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:21:19.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:21:19.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.065 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.420061259999997, 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.066 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.067 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:21:19.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.067 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:19.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:21:19.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:21:19.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:21:19.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:21:19.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:21:19.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:21:19.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:21:19.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:21:19.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:21:19.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.413137569, 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.071 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.072 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.072 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.072 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.072 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.072 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:21:19.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.072 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.072 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.072 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:19.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:19.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:21:19.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:21:19.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:21:19.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:21:19.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:21:19.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:21:19.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:21:19.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:21:19.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:21:19.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:21:19.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:21:19.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:21:19.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:21:19.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:21:19.077 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.077 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.077 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.077 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.077 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:19.077 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.412777985999995, 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:19.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:19.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:21:19.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:21:19.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:21:19.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:21:19.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:21:19.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:21:19.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:21:19.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:21:19.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.082 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 75.412698000000006, 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:21:19.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:21:19.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:19.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:21:19.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:21:19.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:19.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:19.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:19.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:21:19.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:21:19.087 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.087 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.087 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.087 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:19.087 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:19.087 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:19.088 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:19.089 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:19.090 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:19.090 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:19.090 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:19.090+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:19.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:19.091 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:19.091 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:19.091 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:19.092 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:19.092 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:19.092 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:19.093 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:19.094 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:19.094 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:19.106 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:19.106 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:19.190 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:19.191+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:19.202 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:19.202 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:19.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:19.290 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:19.292+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:19.321 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:19.321 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:19.330 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:19.392 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:19.393+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:19.438 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:19.438 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:19.447 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:19.482 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T07:21:19.493 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:19.495+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:19.575 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:19.575 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:19.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:19.593 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:19.594+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:19.693 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:19.694 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:19.694 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:19.695+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:19.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:19.795 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:19.796+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:19.851 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:19.851 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:19.859 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:21:19.900 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:19.902+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:19.996 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:19.998+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:20.007 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:20.007 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:20.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:21:20.099 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:20.101+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:20.163 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:20.163 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:20.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:21:20.214 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:20.215+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:20.300 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:20.302+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:20.320 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:20.320 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:20.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:20.401 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:20.403+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:20.439 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:20.439 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:20.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is set 2024-09-12T07:21:20.502 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:20.503+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:20.570 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:20.570 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:20.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:20.603 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:20.604+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:20.689 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:20.689 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:20.697 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:20.703 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:20.704+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:20.803 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:20.805+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:20.811 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:20.811 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:20.820 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:20.904 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:20.906+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:20.968 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:20.968 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:20.976 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:21.019 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:21.021+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:21.082 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:21.082 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:21.089 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 6, 7, 3] out_osds: [1, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:21:21.089 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:21:21.089 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 2, 4, 6, 7, 3] 2024-09-12T07:21:21.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-12T07:21:21.091 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:21.118 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:21.131 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:21.133+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:21.207 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:21.208+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:21.238 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:21.238 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:21.247 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:21.264 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:21.283 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:21.300 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:21.300 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:21.308 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:21.310+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:21.316 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:21.332 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:21.350 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:21.350 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:21.355 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:21.355 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:21.355 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:21.355 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:21.355 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:21.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:21.367 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:21.367 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:21.384 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:21.415 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-12T07:21:21.417+0000 7f0757424640 -1 mon.a@0(leader).osd e117 definitely_dead 0 2024-09-12T07:21:21.415 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:21.417+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:marked out osd.4. 2024-09-12T07:21:21.517 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:21.518 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:21.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:21.518 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:21.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.518 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.518 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:21.518 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.037072961000007, 2024-09-12T07:21:21.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:21:21.518 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.518 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.518 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:21:21.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.037046081, 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:21:21.523 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:21:21.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.037022061000002, 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:21:21.527 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:21:21.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.530 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.531 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.036994414999995, 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.532 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:21:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:21:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:21:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:21:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:21:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:21:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:21:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:21:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.036970952999994, 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.036948425999995, 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.545 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.036926622999999, 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:21:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:21:21.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.030292842999998, 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.550 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:21:21.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:21:21.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:21:21.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:21:21.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:21:21.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:21:21.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:21:21.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:21:21.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.030272835000005, 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.556 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.556 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.556 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.556 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.556 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:21:21.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.556 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:21.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:21:21.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:21:21.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.559 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.023789803, 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:21:21.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:21:21.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.023754515999997, 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:21:21.564 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:21:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:21:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:21:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:21:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:21:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:21:21.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:21:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.023731882999996, 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.569 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.573 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.023677828000004, 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:21.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.023658886999996, 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.578 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:21:21.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.582 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.02364695, 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.583 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:21:21.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:21:21.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.023635334999994, 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:21:21.587 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:21:21.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:21:21.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.016622726999998, 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:21:21.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.016599012, 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:21:21.596 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:21:21.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.016468154999998, 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:21:21.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:21:21.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:21:21.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:21:21.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:21:21.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 78.052797365000004, 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:21:21.605 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:21:21.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:21.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:21.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:21:21.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:21:21.611 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.611 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.611 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.611 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:21.611 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:21.611 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:21.612 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:21.540+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:21.613 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:21.613 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:21.613+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:21.713 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:21.715+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:21.719 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:21.719 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:21.719 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:21.719 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:21.728 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:21.814 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:21.816+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:21.875 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:21.875 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:21.875 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:21.875 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:21.875 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:21.883 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:21.915 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:21.917+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:22.016 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:22.018+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:22.032 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:22.032 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:22.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:22.032 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:22.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.032 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:21:22.032 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:22.032 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 89.503142982, 2024-09-12T07:21:22.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:21:22.032 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:22.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:21:22.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:21:22.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:21:22.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.651345316000004, 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.038 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:22.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:21:22.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.651222039000004, 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:21:22.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:21:22.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.651018648000004, 2024-09-12T07:21:22.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:21:22.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:21:22.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:21:22.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:21:22.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:21:22.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.650882820999996, 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.053 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:22.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.650556164999998, 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:21:22.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.650517277000006, 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.062 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.063 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.063 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.063 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:21:22.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.063 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:22.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:21:22.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:21:22.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:21:22.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:21:22.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:21:22.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:21:22.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.650150402999998, 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.067 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.068 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.068 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.068 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:21:22.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.068 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:22.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:21:22.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:21:22.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:21:22.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:22.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:21:22.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:21:22.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:21:22.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:21:22.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.650015546999995, 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.072 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.073 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:21:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.649645656000004, 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:21:22.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.078 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:21:22.080 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:22.081 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.649455458000006, 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:22.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:21:22.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.649235379000004, 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.086 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:22.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.090 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.649091713999994, 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:21:22.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.648865541999996, 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.095 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.096 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:21:22.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.648665183999995, 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:21:22.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:21:22.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.648257301000001, 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.104 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:22.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.647791419000001, 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:21:22.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.646189208999999, 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.113 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:21:22.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.645655173999998, 2024-09-12T07:21:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:21:22.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:21:22.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:21:22.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:21:22.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:21:22.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.389457104000002, 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:21:22.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.126 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.127 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.127 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:22.128 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:22.128 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:22.129+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:22.229 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:22.230+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:22.276 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:22.276 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:22.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.276 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-12T07:21:22.276 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:22.276 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 0.78521668600000005, 2024-09-12T07:21:22.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.78523745599999994, 2024-09-12T07:21:22.276 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.276 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.276 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "waiting for peered", 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492882+0000", 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492888+0000", 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:21:22.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492904+0000", 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5628e-05 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492912+0000", 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4039999999999995e-06 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492962+0000", 2024-09-12T07:21:22.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9425999999999997e-05 2024-09-12T07:21:22.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:22.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492969+0000", 2024-09-12T07:21:22.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1729999999999998e-06 2024-09-12T07:21:22.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.280 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.280 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T07:21:22.280 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 1 2024-09-12T07:21:22.280 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:22.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:22.368 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:22.370+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:22.431 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:22.432+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:22.432 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:22.432 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:22.432 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:22.432 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:22.432 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:22.441 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:22.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:21:22.534 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:22.536+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:22.593 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045479530999998, 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:22.598 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045458994000001, 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:22.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:21:22.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:21:22.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:21:22.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:21:22.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045438990999997, 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.604 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.604 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.604 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.604 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:21:22.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.604 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:22.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:21:22.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:21:22.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045419042000006, 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:22.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:21:22.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:21:22.612 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.612 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.612 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.612 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.612 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:22.612 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045398173999999, 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:21:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:21:22.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045376473000005, 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.617 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:21:22.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.621 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045356033000004, 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:21:22.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045328311999995, 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.626 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:21:22.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:21:22.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:21:22.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:21:22.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:21:22.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:21:22.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:21:22.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:21:22.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:21:22.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:21:22.630 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.630 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045305377000005, 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:21:22.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045285077000003, 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.635 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.639 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045264801000002, 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:22.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:22.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:22.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:21:22.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045244642, 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:21:22.644 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.645 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.645 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.645 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.645 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.645 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:21:22.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.645 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.645 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:22.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:21:22.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045107443999996, 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.649 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.651 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:21:22.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.045084795999998, 2024-09-12T07:21:22.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:21:22.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.657 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68088 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 393216~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.038642037000002, 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10346986399999999, 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.658 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68088 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555921+0000", 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555923+0000", 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.891e-06 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555929+0000", 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0549999999999996e-06 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.660 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555934+0000", 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6419999999999998e-06 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659060+0000", 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103125862 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659139+0000", 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9450000000000007e-05 2024-09-12T07:21:22.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659391+0000", 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025158400000000001 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.038622810000007, 2024-09-12T07:21:22.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:21:22.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:21:22.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:21:22.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:21:22.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.038393173000003, 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.667 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.669 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:21:22.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:21:22.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:21:22.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:21:22.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:21:22.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:21:22.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.031469482000006, 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:21:22.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:21:22.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:21:22.675 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.031109899, 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.677 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.677 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:21:22.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.677 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:22.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:21:22.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:21:22.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:21:22.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:21:22.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:21:22.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:21:22.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:22.681 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.031029912999998, 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:22.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:22.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:21:22.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:22.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:22.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:21:22.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:21:22.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:22.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:22.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:21:22.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:21:22.686 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.686 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.686 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.686 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:22.686 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:22.686 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:22.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:22.688 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:22.689+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:22.788 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:22.790+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:22.795 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:22.795 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:22.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:22.889 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:22.891+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:22.952 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:22.952 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:22.952 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:22.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:23.003 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:23.005+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:23.069 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:148945 3.10 3:0cac0a45:::rbd_data.1160161ce0c8.000000000000009d:head [delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-12T07:21:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.636035601000003, 2024-09-12T07:21:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89878390500000005, 2024-09-12T07:21:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 148945 2024-09-12T07:21:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433803+0000", 2024-09-12T07:21:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.164e-06 2024-09-12T07:21:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433807+0000", 2024-09-12T07:21:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5000000000000001e-06 2024-09-12T07:21:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433816+0000", 2024-09-12T07:21:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1859999999999998e-06 2024-09-12T07:21:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433822+0000", 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5300000000000004e-06 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433886+0000", 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4510000000000004e-05 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433896+0000", 2024-09-12T07:21:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0012e-05 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329359+0000", 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89546257399999996 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329585+0000", 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022641800000000001 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329712+0000", 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126339 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.330704+0000", 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099219900000000003 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332513+0000", 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001808689 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332544+0000", 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1186999999999998e-05 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332585+0000", 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1597000000000002e-05 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170165 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1101824~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.508474680999999, 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073287456000000001, 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170165 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561363+0000", 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.96e-07 2024-09-12T07:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561365+0000", 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.181e-06 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561371+0000", 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561377+0000", 2024-09-12T07:21:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.536e-06 2024-09-12T07:21:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634513+0000", 2024-09-12T07:21:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073136229999999997 2024-09-12T07:21:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634548+0000", 2024-09-12T07:21:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5206999999999997e-05 2024-09-12T07:21:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634650+0000", 2024-09-12T07:21:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101678 2024-09-12T07:21:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170166 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.508441340999999, 2024-09-12T07:21:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073346145000000001, 2024-09-12T07:21:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170166 2024-09-12T07:21:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561399+0000", 2024-09-12T07:21:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4049999999999998e-06 2024-09-12T07:21:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561405+0000", 2024-09-12T07:21:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5370000000000003e-06 2024-09-12T07:21:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561410+0000", 2024-09-12T07:21:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2889999999999997e-06 2024-09-12T07:21:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634666+0000", 2024-09-12T07:21:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073255526000000001 2024-09-12T07:21:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634701+0000", 2024-09-12T07:21:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5197999999999997e-05 2024-09-12T07:21:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634742+0000", 2024-09-12T07:21:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0757999999999998e-05 2024-09-12T07:21:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170167 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1183744~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.50841885, 2024-09-12T07:21:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073410903, 2024-09-12T07:21:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170167 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561419+0000", 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3000000000000002e-07 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561421+0000", 2024-09-12T07:21:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1059999999999998e-06 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561427+0000", 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0970000000000002e-06 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561432+0000", 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1120000000000004e-06 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634756+0000", 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073324571000000005 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634790+0000", 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3081999999999999e-05 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634829+0000", 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9604999999999998e-05 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170168 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1224704~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.5083986, 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073476793999999998, 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170168 2024-09-12T07:21:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561439+0000", 2024-09-12T07:21:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561441+0000", 2024-09-12T07:21:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8920000000000001e-06 2024-09-12T07:21:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561447+0000", 2024-09-12T07:21:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2360000000000001e-06 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561452+0000", 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9810000000000003e-06 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634843+0000", 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073391434000000005 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634876+0000", 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2869e-05 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634915+0000", 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9001999999999997e-05 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170169 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1265664~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.508378722, 2024-09-12T07:21:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073551819000000004, 2024-09-12T07:21:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170169 2024-09-12T07:21:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561459+0000", 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.77e-07 2024-09-12T07:21:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561460+0000", 2024-09-12T07:21:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.733e-06 2024-09-12T07:21:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561468+0000", 2024-09-12T07:21:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561474+0000", 2024-09-12T07:21:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4829999999999999e-06 2024-09-12T07:21:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634930+0000", 2024-09-12T07:21:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073456244000000004 2024-09-12T07:21:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634969+0000", 2024-09-12T07:21:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9687000000000001e-05 2024-09-12T07:21:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635010+0000", 2024-09-12T07:21:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0685000000000001e-05 2024-09-12T07:21:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170170 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1306624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.508356327, 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073616897000000001, 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170170 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2e-07 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561483+0000", 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0619999999999999e-06 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561491+0000", 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7880000000000007e-06 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561497+0000", 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5480000000000001e-06 2024-09-12T07:21:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635025+0000", 2024-09-12T07:21:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073528643000000005 2024-09-12T07:21:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635058+0000", 2024-09-12T07:21:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3077999999999997e-05 2024-09-12T07:21:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635098+0000", 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170171 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1347584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.508333562000001, 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073682407000000005, 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170171 2024-09-12T07:21:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561506+0000", 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.441e-06 2024-09-12T07:21:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561535+0000", 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8827999999999999e-05 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561542+0000", 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7100000000000001e-06 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635112+0000", 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073569967999999999 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635146+0000", 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4413999999999999e-05 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635186+0000", 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9625000000000001e-05 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170172 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.508286207999999, 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073720519999999998, 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170172 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1500000000000005e-07 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561554+0000", 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561563+0000", 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7760000000000003e-06 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561568+0000", 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.305e-06 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635200+0000", 2024-09-12T07:21:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073632141999999998 2024-09-12T07:21:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635233+0000", 2024-09-12T07:21:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.239e-05 2024-09-12T07:21:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635271+0000", 2024-09-12T07:21:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8688e-05 2024-09-12T07:21:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170173 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1429504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.508261620000001, 2024-09-12T07:21:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07378113, 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170173 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561576+0000", 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999998e-07 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561578+0000", 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0389999999999999e-06 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561587+0000", 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0999999999999993e-06 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561594+0000", 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.844e-06 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635286+0000", 2024-09-12T07:21:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073691799000000002 2024-09-12T07:21:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635318+0000", 2024-09-12T07:21:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2240999999999998e-05 2024-09-12T07:21:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635357+0000", 2024-09-12T07:21:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8726000000000002e-05 2024-09-12T07:21:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170174 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1470464~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.508233429000001, 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073841247999999998, 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170174 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:23.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:21:23.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561607+0000", 2024-09-12T07:21:23.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:23.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561613+0000", 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3339999999999998e-06 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561618+0000", 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.203e-06 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635371+0000", 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073752811000000001 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635404+0000", 2024-09-12T07:21:23.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3250000000000002e-05 2024-09-12T07:21:23.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635445+0000", 2024-09-12T07:21:23.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0741000000000002e-05 2024-09-12T07:21:23.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.124 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170175 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1511424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.508211808, 2024-09-12T07:21:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073939900000000003, 2024-09-12T07:21:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170175 2024-09-12T07:21:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561626+0000", 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561632+0000", 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8290000000000002e-06 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561642+0000", 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1209999999999996e-06 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561647+0000", 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5119999999999999e-06 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635459+0000", 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073811962999999994 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635521+0000", 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2323000000000006e-05 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635565+0000", 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3763000000000001e-05 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170176 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1552384~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.129 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.508174757999999, 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.074029015000000004, 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170176 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:23.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561663+0000", 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561665+0000", 2024-09-12T07:21:23.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561672+0000", 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4930000000000003e-06 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561677+0000", 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3159999999999996e-06 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635587+0000", 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073909937999999994 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635629+0000", 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2555999999999997e-05 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635691+0000", 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1946000000000006e-05 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.133 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170183 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1757184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.506220859000001, 2024-09-12T07:21:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073321546000000001, 2024-09-12T07:21:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170183 2024-09-12T07:21:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563617+0000", 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999999e-07 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563619+0000", 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.153e-06 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563625+0000", 2024-09-12T07:21:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3160000000000002e-06 2024-09-12T07:21:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563632+0000", 2024-09-12T07:21:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9820000000000002e-06 2024-09-12T07:21:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636752+0000", 2024-09-12T07:21:23.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073120428000000001 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636838+0000", 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5177000000000001e-05 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636938+0000", 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010008 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.138 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.139 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170185 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1798144~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.139 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:23.139 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.505903361, 2024-09-12T07:21:23.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073255606000000001, 2024-09-12T07:21:23.139 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.139 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170185 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5600000000000005e-07 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563989+0000", 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4993e-05 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563998+0000", 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8710000000000003e-06 2024-09-12T07:21:23.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564005+0000", 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8449999999999997e-06 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636991+0000", 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072986350000000005 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637076+0000", 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4248999999999999e-05 2024-09-12T07:21:23.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637189+0000", 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011364199999999999 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170186 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1839104~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.505819803, 2024-09-12T07:21:23.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073312293000000001, 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170186 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564018+0000", 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.99e-07 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564021+0000", 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7410000000000001e-06 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564027+0000", 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5119999999999997e-06 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564033+0000", 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4970000000000001e-06 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637230+0000", 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073197613999999994 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637271+0000", 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1001000000000003e-05 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637330+0000", 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8328999999999997e-05 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.147 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170189 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1921024~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.505484673, 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073393609999999998, 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170189 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.148 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564353+0000", 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9500000000000003e-07 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564355+0000", 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1409999999999999e-06 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564362+0000", 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6279999999999999e-06 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564367+0000", 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5520000000000004e-06 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637563+0000", 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073196285 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637641+0000", 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7051000000000005e-05 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637746+0000", 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010545800000000001 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170191 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1961984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.505201711, 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073246585000000003, 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170191 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564636+0000", 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6400000000000002e-07 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564639+0000", 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.399e-06 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564646+0000", 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0999999999999998e-06 2024-09-12T07:21:23.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564653+0000", 2024-09-12T07:21:23.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9659999999999999e-06 2024-09-12T07:21:23.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637785+0000", 2024-09-12T07:21:23.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073131357999999994 2024-09-12T07:21:23.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637826+0000", 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1742000000000001e-05 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637882+0000", 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5455999999999999e-05 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.157 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170192 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 2002944~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.157 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:23.157 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.505171697, 2024-09-12T07:21:23.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073448428999999996, 2024-09-12T07:21:23.157 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.157 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.157 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.157 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.157 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170192 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564666+0000", 2024-09-12T07:21:23.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:21:23.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564668+0000", 2024-09-12T07:21:23.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:23.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564675+0000", 2024-09-12T07:21:23.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2230000000000004e-06 2024-09-12T07:21:23.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564680+0000", 2024-09-12T07:21:23.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3210000000000004e-06 2024-09-12T07:21:23.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637923+0000", 2024-09-12T07:21:23.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073243483999999998 2024-09-12T07:21:23.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638011+0000", 2024-09-12T07:21:23.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7149000000000002e-05 2024-09-12T07:21:23.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638114+0000", 2024-09-12T07:21:23.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103278 2024-09-12T07:21:23.161 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170198 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 2084864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.504608828, 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073259037999999999, 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:23.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170198 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:23.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565229+0000", 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0200000000000002e-07 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565231+0000", 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2330000000000001e-06 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565238+0000", 2024-09-12T07:21:23.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.596e-06 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565243+0000", 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.31e-06 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638324+0000", 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073080720000000002 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638391+0000", 2024-09-12T07:21:23.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7037000000000002e-05 2024-09-12T07:21:23.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:23.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:23.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:23.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638487+0000", 2024-09-12T07:21:23.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6639999999999996e-05 2024-09-12T07:21:23.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.166 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:23.166 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:23.166 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:23.168 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:23.168 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:23.169+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:23.268 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:23.269+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:23.314 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:23.315 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:23.315 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:23.315 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:23.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:23.369 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:23.371+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:23.396 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:23.471 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:23.473+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:23.472 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:23.472 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:23.472 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:23.472 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:23.472 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:23.481 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:23.485 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:21:23.566 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:23.567 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:23.572 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:23.573+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:23.591 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:23.619 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:23.619 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:23.643 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:23.643 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:23.649 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:23.649 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:23.661 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:21:23.669 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:23.670 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:23.692 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:23.693 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:23.694+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:23.720 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:23.745 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:23.745 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:23.773 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:23.774+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:23.825 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:23.825 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:23.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:21:23.880 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:23.882+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:23.974 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:23.976+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:23.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:21:24.075 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:24.077+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:24.139 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:24.140 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:24.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:24.192 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:24.194+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:24.259 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:24.260 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:24.270 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:24.276 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:24.277+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:24.377 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:24.379+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:24.437 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:24.438 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:24.447 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:24.478 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:24.480+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:24.579 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:24.580+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:24.600 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:24.601 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:24.609 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:24.679 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:24.681+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:24.725 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:24.725 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:24.734 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:24.780 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:24.782+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:24.881 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:24.883+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:24.883 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:24.883 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:24.892 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:24.982 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:24.984+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:24.998 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:24.998 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:25.007 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:25.083 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:25.085+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:25.113 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:25.113 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:25.122 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:25.206 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:25.208+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:25.271 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:25.271 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:25.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:25.320 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:25.322+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:25.386 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:25.388+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:25.426 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:25.427 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:25.435 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:25.487 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:25.489+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:25.524 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:21:25.583 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:25.583 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:25.588 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:25.589+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:25.592 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:25.689 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:25.691+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:25.698 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:25.698 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:25.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:25.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:25.791 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:25.793+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:25.854 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:25.854 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:25.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:25.887 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:25.890 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:25.892+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:25.920 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:25.921 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:25.936 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:25.937 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:25.955 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:25.955 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:25.971 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:25.972 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:25.988 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:25.989 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:25.992 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:25.993+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:26.005 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:26.005 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:26.012 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:26.012 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:26.012 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:26.012 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:26.020 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:26.093 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:26.095+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:26.127 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:26.127 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:26.127 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:26.127 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:26.127 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:21:26.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:26.194 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:26.196+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:26.249 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:26.250 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:26.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:26.250 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:26.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.250 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.250 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:26.250 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701451227000007, 2024-09-12T07:21:26.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:21:26.250 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.250 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:21:26.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:21:26.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:21:26.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:21:26.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:21:26.254 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.254 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701430689999995, 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:21:26.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.256 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:26.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:26.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:21:26.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:21:26.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701410687000006, 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:21:26.259 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:21:26.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:21:26.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701390738000001, 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:21:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:21:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:21:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:21:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:21:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701369869999994, 2024-09-12T07:21:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:21:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:21:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:21:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:21:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:21:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:21:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:21:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:21:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:21:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:21:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:21:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701348168999999, 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:21:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701327728999999, 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:21:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701300008000004, 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:21:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701277073, 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:21:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701256772999997, 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:21:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:21:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:21:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:21:26.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:21:26.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:21:26.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:21:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:21:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:21:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:21:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:21:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701236496999996, 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:21:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:21:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:21:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:21:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.302 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.302 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:26.302 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701216337999995, 2024-09-12T07:21:26.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:21:26.302 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:21:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:21:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:21:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:21:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:21:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:21:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:21:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701079140000004, 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.307 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:21:26.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:21:26.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:21:26.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:21:26.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:21:26.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:21:26.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:21:26.311 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.311 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.701056492000006, 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:21:26.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:21:26.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:21:26.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:21:26.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:21:26.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.316 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.694594506000001, 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:26.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:21:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:21:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:21:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:21:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:21:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.694364868999997, 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:21:26.321 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:21:26.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:21:26.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.687441178, 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:21:26.325 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:21:26.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:21:26.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:21:26.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:21:26.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:21:26.329 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.329 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.329 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.687081594999995, 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:21:26.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:21:26.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.332 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.687001609000006, 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.334 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:26.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:21:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:21:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:21:26.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.339 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-12T07:21:26.339 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:26.339 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 4.7576543579999999, 2024-09-12T07:21:26.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0005192119999999, 2024-09-12T07:21:26.339 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:26.339 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:26.339 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:21:26.339 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:26.339 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:26.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492882+0000", 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492888+0000", 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492904+0000", 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5628e-05 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492912+0000", 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4039999999999995e-06 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492962+0000", 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9425999999999997e-05 2024-09-12T07:21:26.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492969+0000", 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1729999999999998e-06 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.493388+0000", 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0004188510000001 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.493399+0000", 2024-09-12T07:21:26.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1571e-05 2024-09-12T07:21:26.344 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.344 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.344 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.344 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:26.344 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:26.344 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:26.345 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:26.347+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:26.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:26.446 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:26.448+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:26.454 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:26.454 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:26.454 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:26.454 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:26.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:26.498 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 6, 7, 3] out_osds: [1, 5, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:21:26.499 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:21:26.499 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-12T07:21:26.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T07:21:26.549 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:26.551+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:26.571 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:26.571 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:26.571 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:26.572 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:26.572 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:26.580 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:26.649 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:26.650+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:26.732 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:26.733 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:26.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:26.733 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:26.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.733 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:148945 3.10 3:0cac0a45:::rbd_data.1160161ce0c8.000000000000009d:head [delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-12T07:21:26.733 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:26.733 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 50.298975284999997, 2024-09-12T07:21:26.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89878390500000005, 2024-09-12T07:21:26.733 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:26.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 148945 2024-09-12T07:21:26.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.734 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:26.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:26.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433803+0000", 2024-09-12T07:21:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.164e-06 2024-09-12T07:21:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433807+0000", 2024-09-12T07:21:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5000000000000001e-06 2024-09-12T07:21:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433816+0000", 2024-09-12T07:21:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1859999999999998e-06 2024-09-12T07:21:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433822+0000", 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5300000000000004e-06 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433886+0000", 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4510000000000004e-05 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433896+0000", 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0012e-05 2024-09-12T07:21:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329359+0000", 2024-09-12T07:21:26.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89546257399999996 2024-09-12T07:21:26.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329585+0000", 2024-09-12T07:21:26.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022641800000000001 2024-09-12T07:21:26.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:26.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329712+0000", 2024-09-12T07:21:26.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126339 2024-09-12T07:21:26.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:26.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.330704+0000", 2024-09-12T07:21:26.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099219900000000003 2024-09-12T07:21:26.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:26.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332513+0000", 2024-09-12T07:21:26.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001808689 2024-09-12T07:21:26.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:26.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332544+0000", 2024-09-12T07:21:26.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1186999999999998e-05 2024-09-12T07:21:26.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332585+0000", 2024-09-12T07:21:26.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1597000000000002e-05 2024-09-12T07:21:26.741 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.741 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.741 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.741 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170165 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1101824~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.741 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:26.742 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.171414365, 2024-09-12T07:21:26.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073287456000000001, 2024-09-12T07:21:26.742 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.742 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170165 2024-09-12T07:21:26.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.742 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561363+0000", 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.96e-07 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561365+0000", 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.181e-06 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561371+0000", 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561377+0000", 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.536e-06 2024-09-12T07:21:26.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634513+0000", 2024-09-12T07:21:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073136229999999997 2024-09-12T07:21:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634548+0000", 2024-09-12T07:21:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5206999999999997e-05 2024-09-12T07:21:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634650+0000", 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101678 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170166 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.171381024999999, 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073346145000000001, 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170166 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561399+0000", 2024-09-12T07:21:26.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4049999999999998e-06 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561405+0000", 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5370000000000003e-06 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561410+0000", 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2889999999999997e-06 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634666+0000", 2024-09-12T07:21:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073255526000000001 2024-09-12T07:21:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634701+0000", 2024-09-12T07:21:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5197999999999997e-05 2024-09-12T07:21:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634742+0000", 2024-09-12T07:21:26.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0757999999999998e-05 2024-09-12T07:21:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170167 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1183744~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.171358533999999, 2024-09-12T07:21:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073410903, 2024-09-12T07:21:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170167 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561419+0000", 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3000000000000002e-07 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561421+0000", 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1059999999999998e-06 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561427+0000", 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0970000000000002e-06 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561432+0000", 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1120000000000004e-06 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634756+0000", 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073324571000000005 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634790+0000", 2024-09-12T07:21:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3081999999999999e-05 2024-09-12T07:21:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634829+0000", 2024-09-12T07:21:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9604999999999998e-05 2024-09-12T07:21:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170168 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1224704~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.171338284000001, 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073476793999999998, 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170168 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:26.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561439+0000", 2024-09-12T07:21:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561441+0000", 2024-09-12T07:21:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8920000000000001e-06 2024-09-12T07:21:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561447+0000", 2024-09-12T07:21:26.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2360000000000001e-06 2024-09-12T07:21:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561452+0000", 2024-09-12T07:21:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9810000000000003e-06 2024-09-12T07:21:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634843+0000", 2024-09-12T07:21:26.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073391434000000005 2024-09-12T07:21:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634876+0000", 2024-09-12T07:21:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2869e-05 2024-09-12T07:21:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634915+0000", 2024-09-12T07:21:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9001999999999997e-05 2024-09-12T07:21:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170169 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1265664~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.765 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.171318406000001, 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073551819000000004, 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170169 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561459+0000", 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.77e-07 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561460+0000", 2024-09-12T07:21:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.733e-06 2024-09-12T07:21:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561468+0000", 2024-09-12T07:21:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561474+0000", 2024-09-12T07:21:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4829999999999999e-06 2024-09-12T07:21:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634930+0000", 2024-09-12T07:21:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073456244000000004 2024-09-12T07:21:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634969+0000", 2024-09-12T07:21:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9687000000000001e-05 2024-09-12T07:21:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635010+0000", 2024-09-12T07:21:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0685000000000001e-05 2024-09-12T07:21:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.770 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170170 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1306624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.171296010999999, 2024-09-12T07:21:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073616897000000001, 2024-09-12T07:21:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170170 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2e-07 2024-09-12T07:21:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561483+0000", 2024-09-12T07:21:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0619999999999999e-06 2024-09-12T07:21:26.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561491+0000", 2024-09-12T07:21:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7880000000000007e-06 2024-09-12T07:21:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561497+0000", 2024-09-12T07:21:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5480000000000001e-06 2024-09-12T07:21:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635025+0000", 2024-09-12T07:21:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073528643000000005 2024-09-12T07:21:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635058+0000", 2024-09-12T07:21:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3077999999999997e-05 2024-09-12T07:21:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635098+0000", 2024-09-12T07:21:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:21:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170171 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1347584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.776 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.171273245999998, 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073682407000000005, 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170171 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:21:26.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561506+0000", 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.441e-06 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561535+0000", 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8827999999999999e-05 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561542+0000", 2024-09-12T07:21:26.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7100000000000001e-06 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635112+0000", 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073569967999999999 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635146+0000", 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4413999999999999e-05 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635186+0000", 2024-09-12T07:21:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9625000000000001e-05 2024-09-12T07:21:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170172 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:26.781 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.171225891999999, 2024-09-12T07:21:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073720519999999998, 2024-09-12T07:21:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170172 2024-09-12T07:21:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1500000000000005e-07 2024-09-12T07:21:26.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561554+0000", 2024-09-12T07:21:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-12T07:21:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561563+0000", 2024-09-12T07:21:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7760000000000003e-06 2024-09-12T07:21:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561568+0000", 2024-09-12T07:21:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.305e-06 2024-09-12T07:21:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635200+0000", 2024-09-12T07:21:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073632141999999998 2024-09-12T07:21:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635233+0000", 2024-09-12T07:21:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.239e-05 2024-09-12T07:21:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635271+0000", 2024-09-12T07:21:26.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8688e-05 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170173 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1429504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.171201304, 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07378113, 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170173 2024-09-12T07:21:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561576+0000", 2024-09-12T07:21:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999998e-07 2024-09-12T07:21:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561578+0000", 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0389999999999999e-06 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561587+0000", 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0999999999999993e-06 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561594+0000", 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.844e-06 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635286+0000", 2024-09-12T07:21:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073691799000000002 2024-09-12T07:21:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635318+0000", 2024-09-12T07:21:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2240999999999998e-05 2024-09-12T07:21:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635357+0000", 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8726000000000002e-05 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170174 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1470464~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.171173112999998, 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073841247999999998, 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170174 2024-09-12T07:21:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.793 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561607+0000", 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561613+0000", 2024-09-12T07:21:26.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3339999999999998e-06 2024-09-12T07:21:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561618+0000", 2024-09-12T07:21:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.203e-06 2024-09-12T07:21:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635371+0000", 2024-09-12T07:21:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073752811000000001 2024-09-12T07:21:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635404+0000", 2024-09-12T07:21:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3250000000000002e-05 2024-09-12T07:21:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635445+0000", 2024-09-12T07:21:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0741000000000002e-05 2024-09-12T07:21:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170175 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1511424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.171151492, 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073939900000000003, 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170175 2024-09-12T07:21:26.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561626+0000", 2024-09-12T07:21:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561632+0000", 2024-09-12T07:21:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8290000000000002e-06 2024-09-12T07:21:26.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561642+0000", 2024-09-12T07:21:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1209999999999996e-06 2024-09-12T07:21:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561647+0000", 2024-09-12T07:21:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5119999999999999e-06 2024-09-12T07:21:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635459+0000", 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073811962999999994 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635521+0000", 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2323000000000006e-05 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635565+0000", 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3763000000000001e-05 2024-09-12T07:21:26.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170176 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1552384~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.171114442, 2024-09-12T07:21:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.074029015000000004, 2024-09-12T07:21:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170176 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561663+0000", 2024-09-12T07:21:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:21:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561665+0000", 2024-09-12T07:21:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:21:26.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561672+0000", 2024-09-12T07:21:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4930000000000003e-06 2024-09-12T07:21:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561677+0000", 2024-09-12T07:21:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3159999999999996e-06 2024-09-12T07:21:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635587+0000", 2024-09-12T07:21:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073909937999999994 2024-09-12T07:21:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635629+0000", 2024-09-12T07:21:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2555999999999997e-05 2024-09-12T07:21:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635691+0000", 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1946000000000006e-05 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170183 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1757184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.169160543, 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073321546000000001, 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.808 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170183 2024-09-12T07:21:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563617+0000", 2024-09-12T07:21:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999999e-07 2024-09-12T07:21:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563619+0000", 2024-09-12T07:21:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.153e-06 2024-09-12T07:21:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563625+0000", 2024-09-12T07:21:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3160000000000002e-06 2024-09-12T07:21:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563632+0000", 2024-09-12T07:21:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9820000000000002e-06 2024-09-12T07:21:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636752+0000", 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073120428000000001 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636838+0000", 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5177000000000001e-05 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636938+0000", 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010008 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.813 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170185 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1798144~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.168843044999999, 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073255606000000001, 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170185 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5600000000000005e-07 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563989+0000", 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4993e-05 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563998+0000", 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8710000000000003e-06 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564005+0000", 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8449999999999997e-06 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636991+0000", 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072986350000000005 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637076+0000", 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4248999999999999e-05 2024-09-12T07:21:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637189+0000", 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011364199999999999 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170186 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1839104~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.168759486999999, 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073312293000000001, 2024-09-12T07:21:26.818 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170186 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564018+0000", 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.99e-07 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564021+0000", 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7410000000000001e-06 2024-09-12T07:21:26.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564027+0000", 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5119999999999997e-06 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564033+0000", 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4970000000000001e-06 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637230+0000", 2024-09-12T07:21:26.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073197613999999994 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637271+0000", 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1001000000000003e-05 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637330+0000", 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8328999999999997e-05 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170189 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1921024~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.168424356999999, 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073393609999999998, 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170189 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564353+0000", 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9500000000000003e-07 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564355+0000", 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1409999999999999e-06 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564362+0000", 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6279999999999999e-06 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564367+0000", 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5520000000000004e-06 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637563+0000", 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073196285 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637641+0000", 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7051000000000005e-05 2024-09-12T07:21:26.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637746+0000", 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010545800000000001 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170191 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1961984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.168141394999999, 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073246585000000003, 2024-09-12T07:21:26.827 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170191 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564636+0000", 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6400000000000002e-07 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564639+0000", 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.399e-06 2024-09-12T07:21:26.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564646+0000", 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0999999999999998e-06 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564653+0000", 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9659999999999999e-06 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637785+0000", 2024-09-12T07:21:26.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073131357999999994 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637826+0000", 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1742000000000001e-05 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637882+0000", 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5455999999999999e-05 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170192 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 2002944~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.168111380999999, 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073448428999999996, 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170192 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.832 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564666+0000", 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564668+0000", 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564675+0000", 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2230000000000004e-06 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564680+0000", 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3210000000000004e-06 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637923+0000", 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073243483999999998 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638011+0000", 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7149000000000002e-05 2024-09-12T07:21:26.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638114+0000", 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103278 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170198 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 2084864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.167548512, 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073259037999999999, 2024-09-12T07:21:26.836 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170198 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565229+0000", 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0200000000000002e-07 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565231+0000", 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2330000000000001e-06 2024-09-12T07:21:26.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565238+0000", 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.596e-06 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565243+0000", 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.31e-06 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638324+0000", 2024-09-12T07:21:26.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073080720000000002 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638391+0000", 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7037000000000002e-05 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638487+0000", 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6639999999999996e-05 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.841 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.841 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.841 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:26.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:21:26.844 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 123, 2024-09-12T07:21:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "8412ed58-ac5e-4549-b523-1a58cf9f9429", 2024-09-12T07:21:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "created": "2024-09-12T07:19:10.328057+0000", 2024-09-12T07:21:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-12T07:21:26.128843+0000", 2024-09-12T07:21:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "last_up_change": "2024-09-12T07:19:22.177473+0000", 2024-09-12T07:21:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "last_in_change": "2024-09-12T07:21:21.476884+0000", 2024-09-12T07:21:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T07:21:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_num": 5799936, 2024-09-12T07:21:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_set": [ 2024-09-12T07:21:26.844 INFO:teuthology.orchestra.run.smithi046.stdout: "pglog_hardlimit", 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "purged_snapdirs", 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_deletes", 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "sortbitwise" 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_version": 5, 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_snapshot": "", 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_max": 3, 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd": 8, 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "require_min_compat_client": "luminous", 2024-09-12T07:21:26.845 INFO:teuthology.orchestra.run.smithi046.stdout: "min_compat_client": "luminous", 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "require_osd_release": "squid", 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_crimson": false, 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "pools": [ 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 1, 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": ".mgr", 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "create_time": "2024-09-12T07:19:23.507151+0000", 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_names": "hashpspool", 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "type": 1, 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 2, 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": 1, 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_rule": 0, 2024-09-12T07:21:26.846 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_count": 0, 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_target": 0, 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "is_stretch_pool": false, 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "object_hash": 2, 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscale_mode": "off", 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": 1, 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num": 1, 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num_target": 1, 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_target": 1, 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_pending": 1, 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "last_pg_merge_meta": { 2024-09-12T07:21:26.847 INFO:teuthology.orchestra.run.smithi046.stdout: "source_pgid": "0.0", 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "ready_epoch": 0, 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_started": 0, 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_clean": 0, 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "source_version": "0'0", 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "target_version": "0'0" 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "last_change": "18", 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend": "0", 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "auid": 0, 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_mode": "selfmanaged", 2024-09-12T07:21:26.848 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_seq": 0, 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_epoch": 0, 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_snaps": [], 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps": "[]", 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_bytes": 0, 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_objects": 0, 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tiers": [], 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tier_of": -1, 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "read_tier": -1, 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "write_tier": -1, 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_mode": "none", 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_bytes": 0, 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_objects": 0, 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T07:21:26.849 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_flush_age": 0, 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_evict_age": 0, 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profile": "", 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_params": { 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "none" 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_period": 0, 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_count": 0, 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "use_gmt_hitset": true, 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "min_read_recency_for_promote": 0, 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "min_write_recency_for_promote": 0, 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T07:21:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_search_last_n": 0, 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: "grade_table": [], 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: "stripe_width": 0, 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: "expected_num_objects": 0, 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: "fast_read": false, 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: "options": { 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_max": 32, 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_min": 1 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: "application_metadata": { 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": {} 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: "read_balance": { 2024-09-12T07:21:26.851 INFO:teuthology.orchestra.run.smithi046.stdout: "score_type": "Fair distribution", 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: "score_acting": 6.9000000953674316, 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: "score_stable": 6.9000000953674316, 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: "optimal_score": 0.28999999165534973, 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_acting": 2, 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_stable": 2, 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity_weighted": 0.75648117065429688, 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity_weighted": 0.75999999046325684 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 2, 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": "rbd", 2024-09-12T07:21:26.852 INFO:teuthology.orchestra.run.smithi046.stdout: "create_time": "2024-09-12T07:19:28.593632+0000", 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 8193, 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type": 1, 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 2, 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": 1, 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_rule": 0, 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_count": 0, 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_target": 0, 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "is_stretch_pool": false, 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "object_hash": 2, 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscale_mode": "off", 2024-09-12T07:21:26.853 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": 8, 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num": 8, 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num_target": 8, 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_target": 8, 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_pending": 8, 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "last_pg_merge_meta": { 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "source_pgid": "0.0", 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "ready_epoch": 0, 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_started": 0, 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_clean": 0, 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "source_version": "0'0", 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "target_version": "0'0" 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "last_change": "22", 2024-09-12T07:21:26.854 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend": "0", 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "auid": 0, 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_mode": "selfmanaged", 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_seq": 2, 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_epoch": 22, 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_snaps": [], 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps": "[]", 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_bytes": 0, 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_objects": 0, 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tiers": [], 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tier_of": -1, 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "read_tier": -1, 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "write_tier": -1, 2024-09-12T07:21:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_mode": "none", 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_bytes": 0, 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_objects": 0, 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_flush_age": 0, 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_evict_age": 0, 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profile": "", 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_params": { 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "none" 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_period": 0, 2024-09-12T07:21:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_count": 0, 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: "use_gmt_hitset": true, 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: "min_read_recency_for_promote": 0, 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: "min_write_recency_for_promote": 0, 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_search_last_n": 0, 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: "grade_table": [], 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: "stripe_width": 0, 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: "expected_num_objects": 0, 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: "fast_read": false, 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: "options": {}, 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: "application_metadata": { 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd": {} 2024-09-12T07:21:26.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: "read_balance": { 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: "score_type": "Fair distribution", 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: "score_acting": 2.6500000953674316, 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: "score_stable": 2.6500000953674316, 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: "optimal_score": 0.70999997854232788, 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity_weighted": 0.75648117065429688, 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity_weighted": 0.75999999046325684 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 3, 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": "pool_client.0", 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "create_time": "2024-09-12T07:19:40.555421+0000", 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 8193, 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type": 1, 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 2, 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": 1, 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_rule": 0, 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_count": 0, 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_target": 0, 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T07:21:26.859 INFO:teuthology.orchestra.run.smithi046.stdout: "is_stretch_pool": false, 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "object_hash": 2, 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscale_mode": "off", 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": 32, 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num": 32, 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num_target": 32, 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_target": 32, 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_pending": 32, 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "last_pg_merge_meta": { 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "source_pgid": "0.0", 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "ready_epoch": 0, 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_started": 0, 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_clean": 0, 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "source_version": "0'0", 2024-09-12T07:21:26.860 INFO:teuthology.orchestra.run.smithi046.stdout: "target_version": "0'0" 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "last_change": "121", 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend": "0", 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "auid": 0, 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_mode": "selfmanaged", 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_seq": 47, 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_epoch": 121, 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_snaps": [], 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps": "[]", 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_bytes": 0, 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_objects": 0, 2024-09-12T07:21:26.861 INFO:teuthology.orchestra.run.smithi046.stdout: "tiers": [], 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tier_of": -1, 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "read_tier": -1, 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "write_tier": -1, 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_mode": "none", 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_bytes": 0, 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_objects": 0, 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_flush_age": 0, 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_evict_age": 0, 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profile": "", 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_params": { 2024-09-12T07:21:26.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "none" 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_period": 0, 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_count": 0, 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "use_gmt_hitset": true, 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "min_read_recency_for_promote": 0, 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "min_write_recency_for_promote": 0, 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_search_last_n": 0, 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "grade_table": [], 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "stripe_width": 0, 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "expected_num_objects": 0, 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "fast_read": false, 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "options": {}, 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "application_metadata": { 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd": {} 2024-09-12T07:21:26.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: "read_balance": { 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: "score_type": "Fair distribution", 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: "score_acting": 2.4200000762939453, 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: "score_stable": 2.4200000762939453, 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: "optimal_score": 0.70999997854232788, 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_acting": 1.7200000286102295, 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_stable": 1.7200000286102295, 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity_weighted": 0.75648117065429688, 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity_weighted": 0.75999999046325684 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:26.864 INFO:teuthology.orchestra.run.smithi046.stdout: "osds": [ 2024-09-12T07:21:26.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.865 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 0, 2024-09-12T07:21:26.865 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "710c25c3-92d3-4b51-a7fc-670879faa69c", 2024-09-12T07:21:26.865 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:26.865 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:21:26.865 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:21:26.865 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:21:26.865 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:26.865 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:26.865 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:26.865 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 121, 2024-09-12T07:21:26.865 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6816", 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6817", 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.866 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6818", 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6819", 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6822", 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6823", 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6820", 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6821", 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.869 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6817/4081287959", 2024-09-12T07:21:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6819/4081287959", 2024-09-12T07:21:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6823/4081287959", 2024-09-12T07:21:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6821/4081287959", 2024-09-12T07:21:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 1, 2024-09-12T07:21:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "04479191-c52e-4b60-aa0d-3e911f869062", 2024-09-12T07:21:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 0, 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 0, 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 99, 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.871 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6824", 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6825", 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.872 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6826", 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6827", 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6830", 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6831", 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6828", 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6829", 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6825/3980488213", 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6827/3980488213", 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3980488213", 2024-09-12T07:21:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3980488213", 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 2, 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "ba673e51-96aa-424b-a4df-26432d633ebe", 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:26.876 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 121, 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6800", 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.877 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6801", 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6802", 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6803", 2024-09-12T07:21:26.878 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6806", 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.879 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6807", 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6804", 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.880 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6805", 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6801/656802655", 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6803/656802655", 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6807/656802655", 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6805/656802655", 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 3, 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "c516e59c-834c-4fb0-9ff8-bcc80b21d28a", 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 122, 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6808", 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6809", 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6810", 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6811", 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6814", 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6815", 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:26.885 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6812", 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6813", 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.886 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6809/3848260327", 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6811/3848260327", 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6815/3848260327", 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6813/3848260327", 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 4, 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "450a70cd-ebb2-48ed-a766-184f58e7032e", 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 0, 2024-09-12T07:21:26.887 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 0, 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 0.051849365234375, 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 121, 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6816", 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:26.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6817", 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6818", 2024-09-12T07:21:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6819", 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6822", 2024-09-12T07:21:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6823", 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6820", 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6821", 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6817/3464256744", 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6819/3464256744", 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6823/3464256744", 2024-09-12T07:21:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6821/3464256744", 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 5, 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "a4efa333-bf57-45e7-a281-1d3bc9d2e82b", 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 0, 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 0, 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 97, 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6824", 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6825", 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6826", 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6827", 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6830", 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6831", 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6828", 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6829", 2024-09-12T07:21:26.898 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:26.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.898 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.898 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6825/1789414693", 2024-09-12T07:21:26.898 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6827/1789414693", 2024-09-12T07:21:26.898 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6831/1789414693", 2024-09-12T07:21:26.898 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6829/1789414693", 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 6, 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "bcaad352-f9ee-426c-bcbf-39c8d031aae0", 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 0, 2024-09-12T07:21:26.899 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 100, 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6800", 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6801", 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6802", 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:26.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6804", 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6810", 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6813", 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6806", 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:26.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6808", 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6801/2576774646", 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6804/2576774646", 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6813/2576774646", 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6808/2576774646", 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:26.904 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 7, 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "97e5ef36-58a4-4667-9f9e-b8ced5904d69", 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 122, 2024-09-12T07:21:26.905 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6803", 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6805", 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.906 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6807", 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6809", 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:26.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6814", 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6815", 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.908 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.909 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:26.909 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:26.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.909 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:26.909 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6811", 2024-09-12T07:21:26.909 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:26.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.909 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:26.909 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6812", 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6805/368918691", 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6809/368918691", 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6815/368918691", 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6812/368918691", 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_xinfo": [ 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 0, 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:18.862785+0000", 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.911 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 1, 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 65536, 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.059718+0000", 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 2, 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:26.912 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:26.913 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:21:26.913 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:18.575165+0000", 2024-09-12T07:21:26.913 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:26.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.913 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 3, 2024-09-12T07:21:26.913 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:26.913 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:26.913 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:26.913 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:26.913 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:21:26.913 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:18.847666+0000", 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 4, 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 65536, 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.471904+0000", 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 5, 2024-09-12T07:21:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 65536, 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.061712+0000", 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 6, 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:26.915 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.541383+0000", 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 7, 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.072272+0000", 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:26.916 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_upmap": [ 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: "pgid": "3.a", 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: "osds": [ 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: 4, 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: 1 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_upmap_items": [], 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_upmap_primaries": [], 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_temp": [], 2024-09-12T07:21:26.917 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_temp": [], 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklist": {}, 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: "range_blocklist": {}, 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profiles": { 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: "default": { 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-failure-domain": "osd", 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: "k": "2", 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: "m": "1", 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: "plugin": "jerasure", 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: "technique": "reed_sol_van" 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps_queue": [ 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 3, 2024-09-12T07:21:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: "snaps": [ 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: "begin": 38, 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: "length": 2 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: "new_removed_snaps": [], 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: "new_purged_snaps": [], 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_node_flags": {}, 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: "device_class_flags": {}, 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_mode": { 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_mode_enabled": false, 2024-09-12T07:21:26.919 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_bucket_count": 0, 2024-09-12T07:21:26.920 INFO:teuthology.orchestra.run.smithi046.stdout: "degraded_stretch_mode": 0, 2024-09-12T07:21:26.920 INFO:teuthology.orchestra.run.smithi046.stdout: "recovering_stretch_mode": 0, 2024-09-12T07:21:26.920 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_mode_bucket": 0 2024-09-12T07:21:26.920 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:26.920 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:26.921 INFO:tasks.thrashosds.thrasher:j is {'epoch': 123, 'fsid': '8412ed58-ac5e-4549-b523-1a58cf9f9429', 'created': '2024-09-12T07:19:10.328057+0000', 'modified': '2024-09-12T07:21:26.128843+0000', 'last_up_change': '2024-09-12T07:19:22.177473+0000', 'last_in_change': '2024-09-12T07:21:21.476884+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T07:19:23.507151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7564811706542969, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.7599999904632568}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T07:19:28.593632+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.7099999785423279, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.7564811706542969, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.7599999904632568}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T07:19:40.555421+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 121, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.4200000762939453, 'score_stable': 2.4200000762939453, 'optimal_score': 0.7099999785423279, 'raw_score_acting': 1.7200000286102295, 'raw_score_stable': 1.7200000286102295, 'primary_affinity_weighted': 0.7564811706542969, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.7599999904632568}}], 'osds': [{'osd': 0, 'uuid': '710c25c3-92d3-4b51-a7fc-670879faa69c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6817', 'nonce': 4081287959}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6818', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 4081287959}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6822', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6823', 'nonce': 4081287959}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6821', 'nonce': 4081287959}]}, 'public_addr': '172.21.15.46:6817/4081287959', 'cluster_addr': '172.21.15.46:6819/4081287959', 'heartbeat_back_addr': '172.21.15.46:6823/4081287959', 'heartbeat_front_addr': '172.21.15.46:6821/4081287959', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '04479191-c52e-4b60-aa0d-3e911f869062', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6824', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 3980488213}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6827', 'nonce': 3980488213}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3980488213}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6828', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3980488213}]}, 'public_addr': '172.21.15.46:6825/3980488213', 'cluster_addr': '172.21.15.46:6827/3980488213', 'heartbeat_back_addr': '172.21.15.46:6831/3980488213', 'heartbeat_front_addr': '172.21.15.46:6829/3980488213', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'ba673e51-96aa-424b-a4df-26432d633ebe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6801', 'nonce': 656802655}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6802', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 656802655}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6806', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 656802655}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6805', 'nonce': 656802655}]}, 'public_addr': '172.21.15.46:6801/656802655', 'cluster_addr': '172.21.15.46:6803/656802655', 'heartbeat_back_addr': '172.21.15.46:6807/656802655', 'heartbeat_front_addr': '172.21.15.46:6805/656802655', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c516e59c-834c-4fb0-9ff8-bcc80b21d28a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 122, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6809', 'nonce': 3848260327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6810', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 3848260327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6814', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 3848260327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6813', 'nonce': 3848260327}]}, 'public_addr': '172.21.15.46:6809/3848260327', 'cluster_addr': '172.21.15.46:6811/3848260327', 'heartbeat_back_addr': '172.21.15.46:6815/3848260327', 'heartbeat_front_addr': '172.21.15.46:6813/3848260327', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '450a70cd-ebb2-48ed-a766-184f58e7032e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.051849365234375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6816', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6817', 'nonce': 3464256744}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6818', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6819', 'nonce': 3464256744}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6822', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6823', 'nonce': 3464256744}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6820', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6821', 'nonce': 3464256744}]}, 'public_addr': '172.21.15.64:6817/3464256744', 'cluster_addr': '172.21.15.64:6819/3464256744', 'heartbeat_back_addr': '172.21.15.64:6823/3464256744', 'heartbeat_front_addr': '172.21.15.64:6821/3464256744', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a4efa333-bf57-45e7-a281-1d3bc9d2e82b', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6824', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6825', 'nonce': 1789414693}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6826', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6827', 'nonce': 1789414693}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6830', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6831', 'nonce': 1789414693}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6828', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6829', 'nonce': 1789414693}]}, 'public_addr': '172.21.15.64:6825/1789414693', 'cluster_addr': '172.21.15.64:6827/1789414693', 'heartbeat_back_addr': '172.21.15.64:6831/1789414693', 'heartbeat_front_addr': '172.21.15.64:6829/1789414693', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'bcaad352-f9ee-426c-bcbf-39c8d031aae0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6800', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6801', 'nonce': 2576774646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6802', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6804', 'nonce': 2576774646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6810', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6813', 'nonce': 2576774646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6806', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6808', 'nonce': 2576774646}]}, 'public_addr': '172.21.15.64:6801/2576774646', 'cluster_addr': '172.21.15.64:6804/2576774646', 'heartbeat_back_addr': '172.21.15.64:6813/2576774646', 'heartbeat_front_addr': '172.21.15.64:6808/2576774646', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '97e5ef36-58a4-4667-9f9e-b8ced5904d69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 122, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6803', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6805', 'nonce': 368918691}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6807', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6809', 'nonce': 368918691}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6814', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6815', 'nonce': 368918691}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6811', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6812', 'nonce': 368918691}]}, 'public_addr': '172.21.15.64:6805/368918691', 'cluster_addr': '172.21.15.64:6809/368918691', 'heartbeat_back_addr': '172.21.15.64:6815/368918691', 'heartbeat_front_addr': '172.21.15.64:6812/368918691', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:18.862785+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.059718+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:18.575165+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:18.847666+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.471904+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.061712+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.541383+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.072272+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.a', 'osds': [4, 1]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 38, 'length': 2}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T07:21:26.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:21:26.922 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:26.844+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:26.964 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:26.966+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:27.031 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:27.031 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:27.031 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:27.031 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:27.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:27.043 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:27.045+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:27.145 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:27.146+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:27.199 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:27.199 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:27.199 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:27.199 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:27.199 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:27.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:27.244 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:21:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:21:27.246 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:27.247+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:27.257 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-12T07:21:27.132026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1230831843,"num_objects":1504,"num_object_clones":367,"num_object_copies":3008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1504,"num_whiteouts":630,"num_read":249893,"num_read_kb":1181835,"num_write":18820,"num_write_kb":598508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1690,"num_bytes_recovered":220088645,"num_keys_recovered":456,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15383,"ondisk_log_size":15383,"up":82,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":709592,"kb_used_data":474436,"kb_used_omap":7,"kb_used_meta":234936,"kb_avail":471149608,"statfs":{"total":483183820800,"available":482457198592,"internally_reserved":0,"allocated":485822464,"data_stored":573142029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7966,"internal_metadata":240574690},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.643226"},"pg_stats":[{"pgid":"3.1f","version":"117'557","reported_seq":6076,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.492127+0000","last_change":"2024-09-12T07:21:21.485532+0000","last_active":"2024-09-12T07:21:22.492127+0000","last_peered":"2024-09-12T07:21:22.492127+0000","last_clean":"2024-09-12T07:21:22.492127+0000","last_became_active":"2024-09-12T07:21:08.135678+0000","last_became_peered":"2024-09-12T07:21:08.135678+0000","last_unstale":"2024-09-12T07:21:22.492127+0000","last_undegraded":"2024-09-12T07:21:22.492127+0000","last_fullsized":"2024-09-12T07:21:22.492127+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:54:02.664425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000725072,"stat_sum":{"num_bytes":13840385,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":7432,"num_read_kb":29365,"num_write":717,"num_write_kb":8032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":2971697,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.1e","version":"117'396","reported_seq":1904,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.493015+0000","last_change":"2024-09-12T07:21:21.481118+0000","last_active":"2024-09-12T07:21:22.493015+0000","last_peered":"2024-09-12T07:21:22.493015+0000","last_clean":"2024-09-12T07:21:22.493015+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:21:22.493015+0000","last_undegraded":"2024-09-12T07:21:22.493015+0000","last_fullsized":"2024-09-12T07:21:22.493015+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:34.320106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095537900000000002,"stat_sum":{"num_bytes":58732364,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":19482,"num_read_kb":37644,"num_write":362,"num_write_kb":22336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.1d","version":"117'430","reported_seq":21007,"reported_epoch":117,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:20.562013+0000","last_change":"2024-09-12T07:21:22.492881+0000","last_active":"2024-09-12T07:21:22.492881+0000","last_peered":"2024-09-12T07:21:20.562013+0000","last_clean":"2024-09-12T07:21:20.562013+0000","last_became_active":"2024-09-12T07:21:09.136511+0000","last_became_peered":"2024-09-12T07:21:09.136511+0000","last_unstale":"2024-09-12T07:21:20.562013+0000","last_undegraded":"2024-09-12T07:21:20.562013+0000","last_fullsized":"2024-09-12T07:21:20.562013+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080674200000000005,"stat_sum":{"num_bytes":41834046,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":22,"num_read":20924,"num_read_kb":39045,"num_write":493,"num_write_kb":15255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":10725460,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":3,"acting_primary":4,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"3.1c","version":"117'495","reported_seq":3241,"reported_epoch":117,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:20.533640+0000","last_change":"2024-09-12T07:21:22.491927+0000","last_active":"2024-09-12T07:21:22.491927+0000","last_peered":"2024-09-12T07:21:20.533640+0000","last_clean":"2024-09-12T07:21:20.533640+0000","last_became_active":"2024-09-12T07:21:08.136562+0000","last_became_peered":"2024-09-12T07:21:08.136562+0000","last_unstale":"2024-09-12T07:21:20.533640+0000","last_undegraded":"2024-09-12T07:21:20.533640+0000","last_fullsized":"2024-09-12T07:21:20.533640+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086335000000000003,"stat_sum":{"num_bytes":43084583,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":3523,"num_read_kb":17038,"num_write":665,"num_write_kb":11617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":4528436,"num_keys_recovered":90,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,7],"up_primary":7,"acting_primary":4,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"3.1b","version":"111'358","reported_seq":21114,"reported_epoch":118,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:21.485873+0000","last_change":"2024-09-12T07:21:22.492544+0000","last_active":"2024-09-12T07:21:21.485873+0000","last_peered":"2024-09-12T07:21:20.510780+0000","last_clean":"2024-09-12T07:21:20.510780+0000","last_became_active":"2024-09-12T07:19:42.454990+0000","last_became_peered":"2024-09-12T07:19:42.454990+0000","last_unstale":"2024-09-12T07:21:21.485873+0000","last_undegraded":"2024-09-12T07:21:21.485873+0000","last_fullsized":"2024-09-12T07:21:21.485873+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028559029999999999,"stat_sum":{"num_bytes":44552147,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":21456,"num_read_kb":60702,"num_write":365,"num_write_kb":15703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"3.1a","version":"117'325","reported_seq":3960,"reported_epoch":118,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:21.488417+0000","last_change":"2024-09-12T07:21:22.489356+0000","last_active":"2024-09-12T07:21:21.488417+0000","last_peered":"2024-09-12T07:21:20.577503+0000","last_clean":"2024-09-12T07:21:20.577503+0000","last_became_active":"2024-09-12T07:21:02.463266+0000","last_became_peered":"2024-09-12T07:21:02.463266+0000","last_unstale":"2024-09-12T07:21:21.488417+0000","last_undegraded":"2024-09-12T07:21:21.488417+0000","last_fullsized":"2024-09-12T07:21:21.488417+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.002994296,"stat_sum":{"num_bytes":30929928,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":23,"num_read":4171,"num_read_kb":12756,"num_write":301,"num_write_kb":17319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":9678561,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"3.19","version":"117'593","reported_seq":7222,"reported_epoch":118,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:21.480984+0000","last_change":"2024-09-12T07:21:22.492842+0000","last_active":"2024-09-12T07:21:21.480984+0000","last_peered":"2024-09-12T07:21:20.556825+0000","last_clean":"2024-09-12T07:21:20.556825+0000","last_became_active":"2024-09-12T07:21:02.463476+0000","last_became_peered":"2024-09-12T07:21:02.463476+0000","last_unstale":"2024-09-12T07:21:21.480984+0000","last_undegraded":"2024-09-12T07:21:21.480984+0000","last_fullsized":"2024-09-12T07:21:21.480984+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0030980420000000001,"stat_sum":{"num_bytes":27214545,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":8361,"num_read_kb":50743,"num_write":740,"num_write_kb":15043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":4488187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"3.18","version":"117'549","reported_seq":3432,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.493203+0000","last_change":"2024-09-12T07:21:21.480358+0000","last_active":"2024-09-12T07:21:22.493203+0000","last_peered":"2024-09-12T07:21:22.493203+0000","last_clean":"2024-09-12T07:21:22.493203+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:21:22.493203+0000","last_undegraded":"2024-09-12T07:21:22.493203+0000","last_fullsized":"2024-09-12T07:21:22.493203+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:01:51.132693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052604599999999996,"stat_sum":{"num_bytes":50143050,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":13,"num_read":5935,"num_read_kb":52138,"num_write":699,"num_write_kb":22218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.17","version":"116'559","reported_seq":6383,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.492314+0000","last_change":"2024-09-12T07:21:21.485632+0000","last_active":"2024-09-12T07:21:22.492314+0000","last_peered":"2024-09-12T07:21:22.492314+0000","last_clean":"2024-09-12T07:21:22.492314+0000","last_became_active":"2024-09-12T07:21:08.136717+0000","last_became_peered":"2024-09-12T07:21:08.136717+0000","last_unstale":"2024-09-12T07:21:22.492314+0000","last_undegraded":"2024-09-12T07:21:22.492314+0000","last_fullsized":"2024-09-12T07:21:22.492314+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:19.560564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059307399999999997,"stat_sum":{"num_bytes":37670364,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":8532,"num_read_kb":19510,"num_write":541,"num_write_kb":22973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":20211745,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.16","version":"117'343","reported_seq":1573,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.493474+0000","last_change":"2024-09-12T07:21:21.484779+0000","last_active":"2024-09-12T07:21:22.493474+0000","last_peered":"2024-09-12T07:21:22.493474+0000","last_clean":"2024-09-12T07:21:22.493474+0000","last_became_active":"2024-09-12T07:21:08.139506+0000","last_became_peered":"2024-09-12T07:21:08.139506+0000","last_unstale":"2024-09-12T07:21:22.493474+0000","last_undegraded":"2024-09-12T07:21:22.493474+0000","last_fullsized":"2024-09-12T07:21:22.493474+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:29:43.682255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000926457,"stat_sum":{"num_bytes":18453957,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":23,"num_read":1334,"num_read_kb":18532,"num_write":411,"num_write_kb":14413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":10121363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.15","version":"116'275","reported_seq":6625,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.494784+0000","last_change":"2024-09-12T07:21:21.485463+0000","last_active":"2024-09-12T07:21:22.494784+0000","last_peered":"2024-09-12T07:21:22.494784+0000","last_clean":"2024-09-12T07:21:22.494784+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:21:22.494784+0000","last_undegraded":"2024-09-12T07:21:22.494784+0000","last_fullsized":"2024-09-12T07:21:22.494784+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:23:26.888041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079146200000000005,"stat_sum":{"num_bytes":22445643,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":6654,"num_read_kb":52951,"num_write":258,"num_write_kb":14733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.14","version":"109'311","reported_seq":5044,"reported_epoch":118,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:21.484423+0000","last_change":"2024-09-12T07:21:22.493227+0000","last_active":"2024-09-12T07:21:21.484423+0000","last_peered":"2024-09-12T07:21:20.555946+0000","last_clean":"2024-09-12T07:21:20.555946+0000","last_became_active":"2024-09-12T07:19:42.452277+0000","last_became_peered":"2024-09-12T07:19:42.452277+0000","last_unstale":"2024-09-12T07:21:21.484423+0000","last_undegraded":"2024-09-12T07:21:21.484423+0000","last_fullsized":"2024-09-12T07:21:21.484423+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001663503,"stat_sum":{"num_bytes":57888336,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":19,"num_read":5162,"num_read_kb":60622,"num_write":340,"num_write_kb":30242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"2.7","version":"0'0","reported_seq":98,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.494785+0000","last_change":"2024-09-12T07:20:37.329418+0000","last_active":"2024-09-12T07:21:22.494785+0000","last_peered":"2024-09-12T07:21:22.494785+0000","last_clean":"2024-09-12T07:21:22.494785+0000","last_became_active":"2024-09-12T07:20:37.329145+0000","last_became_peered":"2024-09-12T07:20:37.329145+0000","last_unstale":"2024-09-12T07:21:22.494785+0000","last_undegraded":"2024-09-12T07:21:22.494785+0000","last_fullsized":"2024-09-12T07:21:22.494785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:58.555569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"117'538","reported_seq":5220,"reported_epoch":118,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:21.484349+0000","last_change":"2024-09-12T07:21:22.492513+0000","last_active":"2024-09-12T07:21:21.484349+0000","last_peered":"2024-09-12T07:21:20.563432+0000","last_clean":"2024-09-12T07:21:20.563432+0000","last_became_active":"2024-09-12T07:21:02.463760+0000","last_became_peered":"2024-09-12T07:21:02.463760+0000","last_unstale":"2024-09-12T07:21:21.484349+0000","last_undegraded":"2024-09-12T07:21:21.484349+0000","last_fullsized":"2024-09-12T07:21:21.484349+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027097860000000001,"stat_sum":{"num_bytes":20649904,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":22,"num_read":5788,"num_read_kb":22827,"num_write":655,"num_write_kb":11405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":743566,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.497580+0000","last_change":"2024-09-12T07:21:22.497015+0000","last_active":"2024-09-12T07:21:22.497580+0000","last_peered":"2024-09-12T07:21:22.497580+0000","last_clean":"2024-09-12T07:21:22.497580+0000","last_became_active":"2024-09-12T07:21:22.496674+0000","last_became_peered":"2024-09-12T07:21:22.496674+0000","last_unstale":"2024-09-12T07:21:22.497580+0000","last_undegraded":"2024-09-12T07:21:22.497580+0000","last_fullsized":"2024-09-12T07:21:22.497580+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:07:30.092089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"117'638","reported_seq":1845,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.493924+0000","last_change":"2024-09-12T07:21:21.483091+0000","last_active":"2024-09-12T07:21:22.493924+0000","last_peered":"2024-09-12T07:21:22.493924+0000","last_clean":"2024-09-12T07:21:22.493924+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:21:22.493924+0000","last_undegraded":"2024-09-12T07:21:22.493924+0000","last_fullsized":"2024-09-12T07:21:22.493924+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:31:29.065562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001714637,"stat_sum":{"num_bytes":37997370,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":3023,"num_read_kb":14268,"num_write":876,"num_write_kb":11329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":214,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.494987+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:21:22.494987+0000","last_peered":"2024-09-12T07:21:22.494987+0000","last_clean":"2024-09-12T07:21:22.494987+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:21:22.494987+0000","last_undegraded":"2024-09-12T07:21:22.494987+0000","last_fullsized":"2024-09-12T07:21:22.494987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"115'422","reported_seq":3430,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.489025+0000","last_change":"2024-09-12T07:21:21.488269+0000","last_active":"2024-09-12T07:21:22.489025+0000","last_peered":"2024-09-12T07:21:22.489025+0000","last_clean":"2024-09-12T07:21:22.489025+0000","last_became_active":"2024-09-12T07:21:08.136153+0000","last_became_peered":"2024-09-12T07:21:08.136153+0000","last_unstale":"2024-09-12T07:21:22.489025+0000","last_undegraded":"2024-09-12T07:21:22.489025+0000","last_fullsized":"2024-09-12T07:21:22.489025+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:53:17.346133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055228600000000003,"stat_sum":{"num_bytes":31139649,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":4111,"num_read_kb":14354,"num_write":522,"num_write_kb":8797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6163290,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.491800+0000","last_change":"2024-09-12T07:20:59.438050+0000","last_active":"2024-09-12T07:21:22.491800+0000","last_peered":"2024-09-12T07:21:22.491800+0000","last_clean":"2024-09-12T07:21:22.491800+0000","last_became_active":"2024-09-12T07:20:59.437717+0000","last_became_peered":"2024-09-12T07:20:59.437717+0000","last_unstale":"2024-09-12T07:21:22.491800+0000","last_undegraded":"2024-09-12T07:21:22.491800+0000","last_fullsized":"2024-09-12T07:21:22.491800+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:54:52.326656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"117'481","reported_seq":10299,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.493761+0000","last_change":"2024-09-12T07:21:21.482882+0000","last_active":"2024-09-12T07:21:22.493761+0000","last_peered":"2024-09-12T07:21:22.493761+0000","last_clean":"2024-09-12T07:21:22.493761+0000","last_became_active":"2024-09-12T07:21:08.136836+0000","last_became_peered":"2024-09-12T07:21:08.136836+0000","last_unstale":"2024-09-12T07:21:22.493761+0000","last_undegraded":"2024-09-12T07:21:22.493761+0000","last_fullsized":"2024-09-12T07:21:22.493761+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:12.875395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002498742,"stat_sum":{"num_bytes":54883761,"num_objects":59,"num_object_clones":16,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":23,"num_read":11261,"num_read_kb":77259,"num_write":547,"num_write_kb":25401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":22089334,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.3","version":"117'321","reported_seq":3452,"reported_epoch":118,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:21.485100+0000","last_change":"2024-09-12T07:21:22.493602+0000","last_active":"2024-09-12T07:21:21.485100+0000","last_peered":"2024-09-12T07:21:20.565193+0000","last_clean":"2024-09-12T07:21:20.565193+0000","last_became_active":"2024-09-12T07:19:42.454103+0000","last_became_peered":"2024-09-12T07:19:42.454103+0000","last_unstale":"2024-09-12T07:21:21.485100+0000","last_undegraded":"2024-09-12T07:21:21.485100+0000","last_fullsized":"2024-09-12T07:21:21.485100+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0052558869999999999,"stat_sum":{"num_bytes":23845028,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":26,"num_read":2938,"num_read_kb":11980,"num_write":300,"num_write_kb":10468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"2.2","version":"22'2","reported_seq":45,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.491909+0000","last_change":"2024-09-12T07:21:05.136297+0000","last_active":"2024-09-12T07:21:22.491909+0000","last_peered":"2024-09-12T07:21:22.491909+0000","last_clean":"2024-09-12T07:21:22.491909+0000","last_became_active":"2024-09-12T07:21:05.133284+0000","last_became_peered":"2024-09-12T07:21:05.133284+0000","last_unstale":"2024-09-12T07:21:22.491909+0000","last_undegraded":"2024-09-12T07:21:22.491909+0000","last_fullsized":"2024-09-12T07:21:22.491909+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:12.070898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"113'666","reported_seq":2894,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.488808+0000","last_change":"2024-09-12T07:21:21.487990+0000","last_active":"2024-09-12T07:21:22.488808+0000","last_peered":"2024-09-12T07:21:22.488808+0000","last_clean":"2024-09-12T07:21:22.488808+0000","last_became_active":"2024-09-12T07:21:02.466102+0000","last_became_peered":"2024-09-12T07:21:02.466102+0000","last_unstale":"2024-09-12T07:21:22.488808+0000","last_undegraded":"2024-09-12T07:21:22.488808+0000","last_fullsized":"2024-09-12T07:21:22.488808+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:19.445279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048577899999999999,"stat_sum":{"num_bytes":30205412,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":26,"num_read":4911,"num_read_kb":11193,"num_write":963,"num_write_kb":13361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":9483314,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":226,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.488815+0000","last_change":"2024-09-12T07:20:07.183543+0000","last_active":"2024-09-12T07:21:22.488815+0000","last_peered":"2024-09-12T07:21:22.488815+0000","last_clean":"2024-09-12T07:21:22.488815+0000","last_became_active":"2024-09-12T07:20:07.183213+0000","last_became_peered":"2024-09-12T07:20:07.183213+0000","last_unstale":"2024-09-12T07:21:22.488815+0000","last_undegraded":"2024-09-12T07:21:22.488815+0000","last_fullsized":"2024-09-12T07:21:22.488815+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:24.634459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"117'326","reported_seq":9600,"reported_epoch":118,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:21.484668+0000","last_change":"2024-09-12T07:21:22.492174+0000","last_active":"2024-09-12T07:21:21.484668+0000","last_peered":"2024-09-12T07:21:20.509847+0000","last_clean":"2024-09-12T07:21:20.509847+0000","last_became_active":"2024-09-12T07:19:42.451970+0000","last_became_peered":"2024-09-12T07:19:42.451970+0000","last_unstale":"2024-09-12T07:21:21.484668+0000","last_undegraded":"2024-09-12T07:21:21.484668+0000","last_fullsized":"2024-09-12T07:21:21.484668+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021020090000000002,"stat_sum":{"num_bytes":45285524,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":10656,"num_read_kb":59352,"num_write":331,"num_write_kb":27704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"2.0","version":"0'0","reported_seq":220,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.491256+0000","last_change":"2024-09-12T07:20:59.439161+0000","last_active":"2024-09-12T07:21:22.491256+0000","last_peered":"2024-09-12T07:21:22.491256+0000","last_clean":"2024-09-12T07:21:22.491256+0000","last_became_active":"2024-09-12T07:20:59.438835+0000","last_became_peered":"2024-09-12T07:20:59.438835+0000","last_unstale":"2024-09-12T07:21:22.491256+0000","last_undegraded":"2024-09-12T07:21:22.491256+0000","last_fullsized":"2024-09-12T07:21:22.491256+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:29:29.972775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"118'936","reported_seq":6176,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.493532+0000","last_change":"2024-09-12T07:21:21.481711+0000","last_active":"2024-09-12T07:21:22.493532+0000","last_peered":"2024-09-12T07:21:22.493532+0000","last_clean":"2024-09-12T07:21:22.493532+0000","last_became_active":"2024-09-12T07:21:08.138116+0000","last_became_peered":"2024-09-12T07:21:08.138116+0000","last_unstale":"2024-09-12T07:21:22.493532+0000","last_undegraded":"2024-09-12T07:21:22.493532+0000","last_fullsized":"2024-09-12T07:21:22.493532+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:23.037986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016305009999999999,"stat_sum":{"num_bytes":49414846,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":25,"num_read":8599,"num_read_kb":59545,"num_write":1471,"num_write_kb":27432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":14379761,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"2.3","version":"20'1","reported_seq":50,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.488871+0000","last_change":"2024-09-12T07:21:05.139203+0000","last_active":"2024-09-12T07:21:22.488871+0000","last_peered":"2024-09-12T07:21:22.488871+0000","last_clean":"2024-09-12T07:21:22.488871+0000","last_became_active":"2024-09-12T07:21:05.133673+0000","last_became_peered":"2024-09-12T07:21:05.133673+0000","last_unstale":"2024-09-12T07:21:22.488871+0000","last_undegraded":"2024-09-12T07:21:22.488871+0000","last_fullsized":"2024-09-12T07:21:22.488871+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:49:11.334568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":305,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.491152+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:21:22.491152+0000","last_peered":"2024-09-12T07:21:22.491152+0000","last_clean":"2024-09-12T07:21:22.491152+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:21:22.491152+0000","last_undegraded":"2024-09-12T07:21:22.491152+0000","last_fullsized":"2024-09-12T07:21:22.491152+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"116'865","reported_seq":2395,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.493352+0000","last_change":"2024-09-12T07:21:21.480719+0000","last_active":"2024-09-12T07:21:22.493352+0000","last_peered":"2024-09-12T07:21:22.493352+0000","last_clean":"2024-09-12T07:21:22.493352+0000","last_became_active":"2024-09-12T07:21:02.468797+0000","last_became_peered":"2024-09-12T07:21:02.468797+0000","last_unstale":"2024-09-12T07:21:22.493352+0000","last_undegraded":"2024-09-12T07:21:22.493352+0000","last_fullsized":"2024-09-12T07:21:22.493352+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:59.873085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000693762,"stat_sum":{"num_bytes":35566196,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":21,"num_read":4784,"num_read_kb":23009,"num_write":1377,"num_write_kb":24473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":21462395,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.9","version":"114'445","reported_seq":4477,"reported_epoch":118,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:21.488254+0000","last_change":"2024-09-12T07:21:22.489252+0000","last_active":"2024-09-12T07:21:21.488254+0000","last_peered":"2024-09-12T07:21:20.567895+0000","last_clean":"2024-09-12T07:21:20.567895+0000","last_became_active":"2024-09-12T07:19:42.450230+0000","last_became_peered":"2024-09-12T07:19:42.450230+0000","last_unstale":"2024-09-12T07:21:21.488254+0000","last_undegraded":"2024-09-12T07:21:21.488254+0000","last_fullsized":"2024-09-12T07:21:21.488254+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084459499999999996,"stat_sum":{"num_bytes":35040157,"num_objects":55,"num_object_clones":9,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":23,"num_read":6628,"num_read_kb":31550,"num_write":454,"num_write_kb":20437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"3.a","version":"117'459","reported_seq":5841,"reported_epoch":119,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:22.492132+0000","last_change":"2024-09-12T07:21:22.492132+0000","last_active":"2024-09-12T07:21:22.492132+0000","last_peered":"2024-09-12T07:21:20.556204+0000","last_clean":"2024-09-12T07:21:20.556204+0000","last_became_active":"2024-09-12T07:21:02.466980+0000","last_became_peered":"2024-09-12T07:21:02.466980+0000","last_unstale":"2024-09-12T07:21:22.492132+0000","last_undegraded":"2024-09-12T07:21:22.492132+0000","last_fullsized":"2024-09-12T07:21:22.492132+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0037900450000000001,"stat_sum":{"num_bytes":40956369,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":6174,"num_read_kb":48942,"num_write":469,"num_write_kb":25530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":28400640,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":4,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"3.b","version":"118'600","reported_seq":3025,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.493552+0000","last_change":"2024-09-12T07:21:21.481068+0000","last_active":"2024-09-12T07:21:22.493552+0000","last_peered":"2024-09-12T07:21:22.493552+0000","last_clean":"2024-09-12T07:21:22.493552+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:21:22.493552+0000","last_undegraded":"2024-09-12T07:21:22.493552+0000","last_fullsized":"2024-09-12T07:21:22.493552+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:47.280416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001069289,"stat_sum":{"num_bytes":44330222,"num_objects":53,"num_object_clones":16,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":23,"num_read":5677,"num_read_kb":20796,"num_write":679,"num_write_kb":17061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.c","version":"117'360","reported_seq":9309,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.493581+0000","last_change":"2024-09-12T07:21:21.482887+0000","last_active":"2024-09-12T07:21:22.493581+0000","last_peered":"2024-09-12T07:21:22.493581+0000","last_clean":"2024-09-12T07:21:22.493581+0000","last_became_active":"2024-09-12T07:21:08.138106+0000","last_became_peered":"2024-09-12T07:21:08.138106+0000","last_unstale":"2024-09-12T07:21:22.493581+0000","last_undegraded":"2024-09-12T07:21:22.493581+0000","last_fullsized":"2024-09-12T07:21:22.493581+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:53:32.023905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024787099999999999,"stat_sum":{"num_bytes":49911462,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":16,"num_read":10454,"num_read_kb":49201,"num_write":329,"num_write_kb":15912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":18973449,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.d","version":"117'705","reported_seq":2998,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.494859+0000","last_change":"2024-09-12T07:21:21.486698+0000","last_active":"2024-09-12T07:21:22.494859+0000","last_peered":"2024-09-12T07:21:22.494859+0000","last_clean":"2024-09-12T07:21:22.494859+0000","last_became_active":"2024-09-12T07:21:08.137853+0000","last_became_peered":"2024-09-12T07:21:08.137853+0000","last_unstale":"2024-09-12T07:21:22.494859+0000","last_undegraded":"2024-09-12T07:21:22.494859+0000","last_fullsized":"2024-09-12T07:21:22.494859+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:13.675996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025112139999999999,"stat_sum":{"num_bytes":34897949,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":21,"num_read":4880,"num_read_kb":30508,"num_write":1136,"num_write_kb":13969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":7370075,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.e","version":"117'449","reported_seq":4480,"reported_epoch":118,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:21.484784+0000","last_change":"2024-09-12T07:21:22.494103+0000","last_active":"2024-09-12T07:21:21.484784+0000","last_peered":"2024-09-12T07:21:20.560015+0000","last_clean":"2024-09-12T07:21:20.560015+0000","last_became_active":"2024-09-12T07:19:42.452236+0000","last_became_peered":"2024-09-12T07:19:42.452236+0000","last_unstale":"2024-09-12T07:21:21.484784+0000","last_undegraded":"2024-09-12T07:21:21.484784+0000","last_fullsized":"2024-09-12T07:21:21.484784+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055844349999999996,"stat_sum":{"num_bytes":39322702,"num_objects":58,"num_object_clones":19,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":27,"num_read":5704,"num_read_kb":38549,"num_write":541,"num_write_kb":16881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"3.f","version":"116'504","reported_seq":2073,"reported_epoch":118,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:21.486337+0000","last_change":"2024-09-12T07:21:22.494956+0000","last_active":"2024-09-12T07:21:21.486337+0000","last_peered":"2024-09-12T07:21:20.559821+0000","last_clean":"2024-09-12T07:21:20.559821+0000","last_became_active":"2024-09-12T07:19:42.452155+0000","last_became_peered":"2024-09-12T07:19:42.452155+0000","last_unstale":"2024-09-12T07:21:21.486337+0000","last_undegraded":"2024-09-12T07:21:21.486337+0000","last_fullsized":"2024-09-12T07:21:21.486337+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053187699999999996,"stat_sum":{"num_bytes":29997006,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":18,"num_read":3135,"num_read_kb":13235,"num_write":570,"num_write_kb":16507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"3.10","version":"117'283","reported_seq":1432,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.491481+0000","last_change":"2024-09-12T07:21:21.484741+0000","last_active":"2024-09-12T07:21:22.491481+0000","last_peered":"2024-09-12T07:21:22.491481+0000","last_clean":"2024-09-12T07:21:22.491481+0000","last_became_active":"2024-09-12T07:20:37.328982+0000","last_became_peered":"2024-09-12T07:20:37.328982+0000","last_unstale":"2024-09-12T07:21:22.491481+0000","last_undegraded":"2024-09-12T07:21:22.491481+0000","last_fullsized":"2024-09-12T07:21:22.491481+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:48.095767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044776800000000003,"stat_sum":{"num_bytes":54446222,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":23,"num_read":1264,"num_read_kb":26924,"num_write":299,"num_write_kb":18957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.11","version":"117'398","reported_seq":6518,"reported_epoch":118,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:21.484159+0000","last_change":"2024-09-12T07:21:22.491948+0000","last_active":"2024-09-12T07:21:21.484159+0000","last_peered":"2024-09-12T07:21:20.557927+0000","last_clean":"2024-09-12T07:21:20.557927+0000","last_became_active":"2024-09-12T07:19:42.452104+0000","last_became_peered":"2024-09-12T07:19:42.452104+0000","last_unstale":"2024-09-12T07:21:21.484159+0000","last_undegraded":"2024-09-12T07:21:21.484159+0000","last_fullsized":"2024-09-12T07:21:21.484159+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0037788299999999999,"stat_sum":{"num_bytes":22840029,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":6820,"num_read_kb":47136,"num_write":434,"num_write_kb":21532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"28","length":"8"}]},{"pgid":"3.12","version":"111'310","reported_seq":10517,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-12T07:21:22.491987+0000","last_change":"2024-09-12T07:21:21.484435+0000","last_active":"2024-09-12T07:21:22.491987+0000","last_peered":"2024-09-12T07:21:22.491987+0000","last_clean":"2024-09-12T07:21:22.491987+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:21:22.491987+0000","last_undegraded":"2024-09-12T07:21:22.491987+0000","last_fullsized":"2024-09-12T07:21:22.491987+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:54:52.623807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075251099999999998,"stat_sum":{"num_bytes":33597399,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":11141,"num_read_kb":47335,"num_write":349,"num_write_kb":15568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"9"}]},{"pgid":"3.13","version":"112'411","reported_seq":18959,"reported_epoch":118,"state":"remapped+peering","last_fresh":"2024-09-12T07:21:21.484347+0000","last_change":"2024-09-12T07:21:22.492019+0000","last_active":"2024-09-12T07:21:21.484347+0000","last_peered":"2024-09-12T07:21:20.555323+0000","last_clean":"2024-09-12T07:21:20.555323+0000","last_became_active":"2024-09-12T07:19:42.456127+0000","last_became_peered":"2024-09-12T07:19:42.456127+0000","last_unstale":"2024-09-12T07:21:21.484347+0000","last_undegraded":"2024-09-12T07:21:21.484347+0000","last_fullsized":"2024-09-12T07:21:21.484347+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011045180000000001,"stat_sum":{"num_bytes":69124901,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":24,"num_read":18913,"num_read_kb":82771,"num_write":523,"num_write_kb":44270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"28","length":"8"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1230241456,"num_objects":1500,"num_object_clones":367,"num_object_copies":3000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1500,"num_whiteouts":630,"num_read":249827,"num_read_kb":1181780,"num_write":18717,"num_write_kb":596878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1687,"num_bytes_recovered":220088607,"num_keys_recovered":456,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753545216,"data_stored":910192764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15308,"ondisk_log_size":15308,"up":64,"acting":52,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":168512,"kb_used_data":123336,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94203328,"statfs":{"total":96636764160,"available":96464207872,"internally_reserved":0,"allocated":126296064,"data_stored":149186109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126252,"kb_used_data":86740,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94245588,"statfs":{"total":96636764160,"available":96507482112,"internally_reserved":0,"allocated":88821760,"data_stored":111581212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":0,"seq":60129542171,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542170,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200572,"kb_used_data":124968,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94171268,"statfs":{"total":96636764160,"available":96431378432,"internally_reserved":0,"allocated":127967232,"data_stored":146459201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84892,"kb_used_data":45376,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94286948,"statfs":{"total":96636764160,"available":96549834752,"internally_reserved":0,"allocated":46465024,"data_stored":51978374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":60129542172,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542171,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":129364,"kb_used_data":94016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94242476,"statfs":{"total":96636764160,"available":96504295424,"internally_reserved":0,"allocated":96272384,"data_stored":113937133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94269440,"data_stored":112686713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1589248,"data_stored":1562580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45047808,"data_stored":51311673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":126562304,"data_stored":145799168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":171175936,"data_stored":219830320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":103190528,"data_stored":120145442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":87412736,"data_stored":110921160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":124297216,"data_stored":147935708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:21:27.259 INFO:tasks.thrashosds.thrasher:Setting 2.4 to [1, 0, 5, 2] 2024-09-12T07:21:27.259 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.4', '1', '0', '5', '2'] 2024-09-12T07:21:27.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.4 1 0 5 2 2024-09-12T07:21:27.328 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:27.328 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:27.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:27.328 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:27.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.328 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:27.328 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.945674311999994, 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:21:27.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:21:27.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.334 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.882596047999996, 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:21:27.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:21:27.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:27.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.878890963000003, 2024-09-12T07:21:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:21:27.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:27.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 79.880544684, 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:21:27.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116552 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5267456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.861215332, 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034412801999999999, 2024-09-12T07:21:27.351 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116552 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0400000000000002e-07 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467850+0000", 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6529999999999998e-06 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467856+0000", 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467861+0000", 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502080+0000", 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034218947999999999 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502159+0000", 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9121000000000004e-05 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502260+0000", 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100846 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116553 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5472256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.861194632999997, 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034609677999999998, 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116553 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467868+0000", 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6800000000000001e-07 2024-09-12T07:21:27.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467870+0000", 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.773e-06 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467876+0000", 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0079999999999999e-06 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467880+0000", 2024-09-12T07:21:27.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502304+0000", 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423924000000002 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502379+0000", 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5010000000000002e-05 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502477+0000", 2024-09-12T07:21:27.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7942999999999994e-05 2024-09-12T07:21:27.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.360 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.360 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116554 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5308416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.360 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:27.360 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.861175093, 2024-09-12T07:21:27.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034779583000000003, 2024-09-12T07:21:27.360 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116554 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467889+0000", 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7430000000000001e-06 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467895+0000", 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8159999999999999e-06 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467900+0000", 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502507+0000", 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034607252999999998 2024-09-12T07:21:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502580+0000", 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3337999999999995e-05 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502667+0000", 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6468e-05 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116555 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5349376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.861156119, 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034965729000000001, 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116555 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.365 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1899999999999998e-07 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467908+0000", 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5740000000000001e-06 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467914+0000", 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9939999999999997e-06 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467918+0000", 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.622e-06 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502696+0000", 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034777798999999998 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502780+0000", 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3551e-05 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502872+0000", 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.187e-05 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116557 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3506176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.860061705, 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034335510999999999, 2024-09-12T07:21:27.369 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116557 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469001+0000", 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469003+0000", 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:27.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469012+0000", 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8030000000000003e-06 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469019+0000", 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6959999999999999e-06 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503153+0000", 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034134096000000003 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503236+0000", 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2886000000000006e-05 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503336+0000", 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9919000000000003e-05 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116558 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3469312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.373 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:27.374 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.860031423999999, 2024-09-12T07:21:27.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034519743999999998, 2024-09-12T07:21:27.374 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116558 2024-09-12T07:21:27.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469034+0000", 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469042+0000", 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469047+0000", 2024-09-12T07:21:27.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:21:27.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503370+0000", 2024-09-12T07:21:27.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034323201999999997 2024-09-12T07:21:27.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503454+0000", 2024-09-12T07:21:27.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4109000000000004e-05 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503550+0000", 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6009999999999997e-05 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116559 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3551232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.860004592000003, 2024-09-12T07:21:27.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034689659999999997, 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116559 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469058+0000", 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469061+0000", 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.497e-06 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469069+0000", 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2579999999999995e-06 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469074+0000", 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1279999999999999e-06 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503582+0000", 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034508351 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503656+0000", 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3234999999999999e-05 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503747+0000", 2024-09-12T07:21:27.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1638999999999995e-05 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116560 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.859977864000001, 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034902912000000001, 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.383 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116560 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:27.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469085+0000", 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4700000000000001e-07 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469087+0000", 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4760000000000001e-06 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469095+0000", 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2339999999999994e-06 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469100+0000", 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503796+0000", 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034695411000000002 2024-09-12T07:21:27.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503868+0000", 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2032000000000004e-05 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503987+0000", 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119369 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116561 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3592192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.859954465000001, 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035102170000000002, 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116561 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469110+0000", 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1629999999999999e-06 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469116+0000", 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.251e-06 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469121+0000", 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8210000000000001e-06 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504017+0000", 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034895772999999998 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504108+0000", 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1027999999999997e-05 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504210+0000", 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101729 2024-09-12T07:21:27.391 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116562 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.859933621000003, 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035285642999999998, 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116562 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469129+0000", 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469131+0000", 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1550000000000001e-06 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469138+0000", 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5520000000000001e-06 2024-09-12T07:21:27.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469142+0000", 2024-09-12T07:21:27.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9099999999999996e-06 2024-09-12T07:21:27.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504239+0000", 2024-09-12T07:21:27.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.03509669 2024-09-12T07:21:27.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504311+0000", 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1650999999999996e-05 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504414+0000", 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103286 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.396 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116563 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.859912987999998, 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035471691, 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116563 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469153+0000", 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.05e-06 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469158+0000", 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.874e-06 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469163+0000", 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8069999999999999e-06 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504444+0000", 2024-09-12T07:21:27.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035280793999999997 2024-09-12T07:21:27.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504523+0000", 2024-09-12T07:21:27.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9036999999999996e-05 2024-09-12T07:21:27.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504621+0000", 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7743000000000003e-05 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116564 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5435392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.859891877000003, 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035734657000000003, 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116564 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469171+0000", 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469172+0000", 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.832e-06 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469179+0000", 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1449999999999996e-06 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469184+0000", 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504721+0000", 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035536890000000002 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504809+0000", 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7229000000000001e-05 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504905+0000", 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6337999999999998e-05 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116565 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5476352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.405 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.859871210999998, 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035933158, 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116565 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469193+0000", 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.973e-06 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469201+0000", 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3730000000000004e-06 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469206+0000", 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1660000000000002e-06 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504937+0000", 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035731205000000002 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505035+0000", 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7759999999999999e-05 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505124+0000", 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9258000000000005e-05 2024-09-12T07:21:27.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116566 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5271552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.859848624999998, 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036116848999999999, 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116566 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469214+0000", 2024-09-12T07:21:27.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:27.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469216+0000", 2024-09-12T07:21:27.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8759999999999999e-06 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469222+0000", 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.173e-06 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469227+0000", 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3070000000000002e-06 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505154+0000", 2024-09-12T07:21:27.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035926362000000003 2024-09-12T07:21:27.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505239+0000", 2024-09-12T07:21:27.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5261999999999998e-05 2024-09-12T07:21:27.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505330+0000", 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1436999999999999e-05 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116567 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5312512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.859828096000001, 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036305875000000001, 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116567 2024-09-12T07:21:27.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.416 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.58e-07 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469242+0000", 2024-09-12T07:21:27.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:27.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469249+0000", 2024-09-12T07:21:27.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5549999999999999e-06 2024-09-12T07:21:27.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469254+0000", 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9899999999999997e-06 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505361+0000", 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036106923999999999 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505449+0000", 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8531999999999999e-05 2024-09-12T07:21:27.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505540+0000", 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0805999999999998e-05 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116568 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5353472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.859801423, 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036561383000000003, 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116568 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:27.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469263+0000", 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7549999999999999e-06 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469269+0000", 2024-09-12T07:21:27.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7219999999999996e-06 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469273+0000", 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505631+0000", 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036357997000000003 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505711+0000", 2024-09-12T07:21:27.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9331000000000003e-05 2024-09-12T07:21:27.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505822+0000", 2024-09-12T07:21:27.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000111208 2024-09-12T07:21:27.424 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.424 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.424 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.424 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.424 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.424 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:27.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:21:27.426 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:27.427+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:27.526 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:27.528+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:27.575 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:27.575 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:27.575 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:27.576 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:27.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:21:27.627 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:27.628+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:27.727 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:27.728+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:27.731 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:27.731 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:27.731 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:27.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:21:27.828 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:27.830+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:27.888 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:27.888 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:27.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:27.888 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:27.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.888 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77440 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13156352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.888 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:21:27.888 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.643333178000006, 2024-09-12T07:21:27.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051399724000000001, 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77440 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:21:27.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246267+0000", 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082256199999999999 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246271+0000", 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6700000000000002e-06 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246283+0000", 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.207e-05 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246291+0000", 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296645+0000", 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050353872000000001 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296729+0000", 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4215000000000006e-05 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296844+0000", 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114626 2024-09-12T07:21:27.892 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77446 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.642334663, 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051543643, 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77446 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246579+0000", 2024-09-12T07:21:27.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013651399999999999 2024-09-12T07:21:27.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246583+0000", 2024-09-12T07:21:27.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1509999999999997e-06 2024-09-12T07:21:27.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246593+0000", 2024-09-12T07:21:27.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5670000000000001e-06 2024-09-12T07:21:27.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246599+0000", 2024-09-12T07:21:27.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5189999999999998e-06 2024-09-12T07:21:27.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297826+0000", 2024-09-12T07:21:27.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051226381000000001 2024-09-12T07:21:27.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297903+0000", 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6710000000000002e-05 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297986+0000", 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3801000000000006e-05 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77447 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 5783552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.898 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.642167048999994, 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051512637, 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77447 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246812+0000", 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000201506 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246817+0000", 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246828+0000", 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1693e-05 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246836+0000", 2024-09-12T07:21:27.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4429999999999997e-06 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298021+0000", 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051184940999999998 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298067+0000", 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6292000000000003e-05 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298123+0000", 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5828999999999997e-05 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77448 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.641927311000003, 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051506024999999997, 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77448 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246892+0000", 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1854999999999998e-05 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246895+0000", 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3129999999999999e-06 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246904+0000", 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6179999999999995e-06 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246911+0000", 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7939999999999997e-06 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298164+0000", 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051253559999999997 2024-09-12T07:21:27.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298249+0000", 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4549000000000007e-05 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298356+0000", 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107336 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77449 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.641856114999996, 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051582830000000003, 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77449 2024-09-12T07:21:27.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247137+0000", 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021599499999999999 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247142+0000", 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247154+0000", 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1878e-05 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247168+0000", 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4114000000000001e-05 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298391+0000", 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051223249999999998 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298446+0000", 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4366000000000001e-05 2024-09-12T07:21:27.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298504+0000", 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8273000000000003e-05 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77450 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12681216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.641596544999999, 2024-09-12T07:21:27.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051562375000000001, 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77450 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2799999999999996e-07 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247184+0000", 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247192+0000", 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5920000000000003e-06 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247197+0000", 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6720000000000001e-06 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298550+0000", 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051352082 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298641+0000", 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1236000000000005e-05 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298743+0000", 2024-09-12T07:21:27.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102428 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77451 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7061504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.641572431, 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051813773, 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.917 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77451 2024-09-12T07:21:27.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.918 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:21:27.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247335+0000", 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013012599999999999 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247340+0000", 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.544e-06 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247351+0000", 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.111e-05 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247358+0000", 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9859999999999997e-06 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298909+0000", 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051551781999999997 2024-09-12T07:21:27.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298960+0000", 2024-09-12T07:21:27.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0594999999999999e-05 2024-09-12T07:21:27.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299019+0000", 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8629999999999999e-05 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77452 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.641407650999994, 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051892134999999999, 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77452 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247535+0000", 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165508 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247552+0000", 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6514000000000001e-05 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247565+0000", 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3372e-05 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247577+0000", 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299064+0000", 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051486697999999997 2024-09-12T07:21:27.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299150+0000", 2024-09-12T07:21:27.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6549000000000001e-05 2024-09-12T07:21:27.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299262+0000", 2024-09-12T07:21:27.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011135 2024-09-12T07:21:27.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77453 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.641186930000003, 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051901623000000001, 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77453 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247710+0000", 2024-09-12T07:21:27.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011951 2024-09-12T07:21:27.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247713+0000", 2024-09-12T07:21:27.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4539999999999999e-06 2024-09-12T07:21:27.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247720+0000", 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0339999999999999e-06 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247730+0000", 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0030000000000001e-05 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299297+0000", 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051566446000000002 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299382+0000", 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4963000000000006e-05 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299492+0000", 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011018600000000001 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77454 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7618560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.641038382999994, 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051921851999999998, 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77454 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.932 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247900+0000", 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016088699999999999 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247904+0000", 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6950000000000002e-06 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247912+0000", 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7129999999999994e-06 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247921+0000", 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0299999999999999e-06 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299526+0000", 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051604744000000001 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299601+0000", 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5129000000000005e-05 2024-09-12T07:21:27.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299661+0000", 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9654000000000002e-05 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77455 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12746752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.640844514999998, 2024-09-12T07:21:27.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051971254000000001, 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77455 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:21:27.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248181+0000", 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024790000000000001 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248185+0000", 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5079999999999999e-06 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248197+0000", 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.132e-05 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248203+0000", 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6320000000000002e-06 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299705+0000", 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051501342999999998 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299799+0000", 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4340999999999997e-05 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299904+0000", 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010521 2024-09-12T07:21:27.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77456 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7979008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.640562290000005, 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051860577999999997, 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77456 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248545+0000", 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033009700000000001 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248550+0000", 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248560+0000", 2024-09-12T07:21:27.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248568+0000", 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4939999999999997e-06 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299973+0000", 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051405470000000002 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300019+0000", 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5448999999999998e-05 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300076+0000", 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7073000000000001e-05 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.945 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77457 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13172736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.640195962000007, 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051737914000000003, 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77457 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248582+0000", 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248584+0000", 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248593+0000", 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248609+0000", 2024-09-12T07:21:27.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5664000000000001e-05 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300117+0000", 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051507605999999997 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300212+0000", 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4919000000000004e-05 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300319+0000", 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010776399999999999 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77458 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.640153544, 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051863578, 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.950 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77458 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:27.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248627+0000", 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1389999999999999e-06 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248635+0000", 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4019999999999996e-06 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248641+0000", 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1689999999999997e-06 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300359+0000", 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051717889000000003 2024-09-12T07:21:27.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300418+0000", 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8689e-05 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300487+0000", 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9756000000000001e-05 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77459 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13238272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.640127851000003, 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052076504000000003, 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77459 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:21:27.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:21:27.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250611+0000", 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019617879999999999 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250618+0000", 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1829999999999997e-06 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250629+0000", 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0438000000000001e-05 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250648+0000", 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9355e-05 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300528+0000", 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049879252999999998 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300619+0000", 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1910000000000006e-05 2024-09-12T07:21:27.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300726+0000", 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010657700000000001 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77467 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.637922032000006, 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05160754, 2024-09-12T07:21:27.959 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77467 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250856+0000", 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250858+0000", 2024-09-12T07:21:27.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9180000000000001e-06 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250864+0000", 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.066e-06 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250869+0000", 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302283+0000", 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051414591000000003 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302378+0000", 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4283000000000005e-05 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302463+0000", 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5249999999999999e-05 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.963 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77468 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14909440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.637902459000003, 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051742162000000001, 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.964 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77468 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250877+0000", 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.815e-06 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250883+0000", 2024-09-12T07:21:27.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250888+0000", 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302496+0000", 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051607394000000001 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302553+0000", 2024-09-12T07:21:27.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6971000000000001e-05 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302617+0000", 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4418999999999994e-05 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77469 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14974976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.637882279999999, 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965694, 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77469 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:27.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250897+0000", 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9130000000000001e-06 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250903+0000", 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0129999999999999e-06 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250908+0000", 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9679999999999997e-06 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302661+0000", 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051753066 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302755+0000", 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3376999999999997e-05 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302861+0000", 2024-09-12T07:21:27.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105924 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77470 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12828672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.637862603000002, 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052098589000000001, 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.973 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77470 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:27.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250917+0000", 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8190000000000001e-06 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250923+0000", 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250928+0000", 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9420000000000003e-06 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302894+0000", 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965863000000001 2024-09-12T07:21:27.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302955+0000", 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1235999999999994e-05 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303013+0000", 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8167000000000001e-05 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77471 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12894208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.637843056999998, 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052326263999999997, 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77471 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256961+0000", 2024-09-12T07:21:27.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060271209999999999 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256972+0000", 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0755e-05 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256986+0000", 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845e-05 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256996+0000", 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303060+0000", 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.046063661999999998 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303149+0000", 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8949999999999994e-05 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:27.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:27.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:27.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303261+0000", 2024-09-12T07:21:27.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011195 2024-09-12T07:21:27.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.982 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:27.982 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:27.982 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:27.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:27.984 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:27.985+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:28.016 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:28.084 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:28.085+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:28.134 INFO:teuthology.orchestra.run.smithi046.stderr:set 2.4 pg_upmap_items mapping to [1->0,5->2] 2024-09-12T07:21:28.136 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:28.136 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:28.136 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:28.136 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:28.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:21:28.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:28.156 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:28.180 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:28.181 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:28.185 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:28.186+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:28.202 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:28.237 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:28.269 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:28.269 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:28.285 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:28.287+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:28.301 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:28.310 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:28.310 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:28.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:28.334 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:28.353 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:28.387 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:28.388+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:28.444 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:28.444 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:28.452 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:28.497 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:28.499+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:28.562 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:28.562 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:28.571 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:28.589 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:28.591+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:28.690 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:28.692+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:28.716 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:28.716 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:28.725 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:28.791 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:28.793+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:28.872 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:28.872 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:28.882 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:28.892 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:28.894+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:28.989 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:28.989 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:28.993 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:28.995+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:28.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:29.094 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:29.095+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:29.104 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:29.104 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:29.113 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:29.139 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:21:29.198 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:29.200+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:29.229 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:29.229 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:29.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:29.296 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:29.298+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:29.347 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:29.348 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:29.356 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:29.397 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:29.398+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:29.497 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:29.499+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:29.503 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:29.503 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:29.511 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:29.598 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:29.600+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:29.618 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:29.619 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:29.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:29.711 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:29.713+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:29.775 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:29.775 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:29.784 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:29.826 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:29.827+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:29.901 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:29.902+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:29.931 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:29.932 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:29.940 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:30.002 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:30.003+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:30.047 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:30.047 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:30.056 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:30.103 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:30.104+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:30.188 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:30.188 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:30.199 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:30.204 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:30.205+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:30.305 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:30.307+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:30.350 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:30.351 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:30.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:30.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:30.406 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:30.408+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:30.476 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:30.521 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:30.522+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:30.528 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:30.528 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:30.545 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:30.546 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:30.562 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:30.578 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:30.587 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:30.587 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:30.595 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:30.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:30.608 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:30.609+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:30.612 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:30.629 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:30.629 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:30.646 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:30.708 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:30.710+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:30.720 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:30.720 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:30.720 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:30.720 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:30.720 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:30.729 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:30.809 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:30.810+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:30.837 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:30.837 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:30.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:30.837 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:30.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.838 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:30.838 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:30.838 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.454633180000002, 2024-09-12T07:21:30.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:21:30.838 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:30.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:21:30.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.839 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:30.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:21:30.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:21:30.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:21:30.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:21:30.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:21:30.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:21:30.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:21:30.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:21:30.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:21:30.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:21:30.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:21:30.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:30.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:21:30.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:21:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:21:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.391554916000004, 2024-09-12T07:21:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:21:30.845 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.846 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:30.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.846 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:21:30.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.846 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:30.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:30.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:21:30.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:21:30.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:21:30.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:21:30.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:21:30.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:21:30.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:21:30.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.851 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.387849830999997, 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:21:30.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:21:30.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:30.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:30.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:21:30.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:21:30.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:21:30.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:21:30.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.857 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.858 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:21:30.858 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:30.858 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.389503551999994, 2024-09-12T07:21:30.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:21:30.858 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:21:30.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.859 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:30.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:30.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:21:30.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:30.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:21:30.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:21:30.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.863 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.863 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116552 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5267456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.863 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:30.863 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.370174199999994, 2024-09-12T07:21:30.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034412801999999999, 2024-09-12T07:21:30.864 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.864 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116552 2024-09-12T07:21:30.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.864 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:30.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:30.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0400000000000002e-07 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467850+0000", 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6529999999999998e-06 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467856+0000", 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:30.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467861+0000", 2024-09-12T07:21:30.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:21:30.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502080+0000", 2024-09-12T07:21:30.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034218947999999999 2024-09-12T07:21:30.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502159+0000", 2024-09-12T07:21:30.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9121000000000004e-05 2024-09-12T07:21:30.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502260+0000", 2024-09-12T07:21:30.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100846 2024-09-12T07:21:30.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.868 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116553 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5472256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.370153501000004, 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034609677999999998, 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116553 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467868+0000", 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6800000000000001e-07 2024-09-12T07:21:30.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467870+0000", 2024-09-12T07:21:30.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.773e-06 2024-09-12T07:21:30.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467876+0000", 2024-09-12T07:21:30.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0079999999999999e-06 2024-09-12T07:21:30.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467880+0000", 2024-09-12T07:21:30.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-12T07:21:30.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502304+0000", 2024-09-12T07:21:30.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423924000000002 2024-09-12T07:21:30.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502379+0000", 2024-09-12T07:21:30.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5010000000000002e-05 2024-09-12T07:21:30.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502477+0000", 2024-09-12T07:21:30.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7942999999999994e-05 2024-09-12T07:21:30.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.874 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.874 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116554 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5308416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.874 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:30.874 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.370133960999993, 2024-09-12T07:21:30.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034779583000000003, 2024-09-12T07:21:30.875 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.875 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.875 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.875 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.875 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116554 2024-09-12T07:21:30.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.875 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:30.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:30.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467889+0000", 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7430000000000001e-06 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467895+0000", 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8159999999999999e-06 2024-09-12T07:21:30.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467900+0000", 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502507+0000", 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034607252999999998 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502580+0000", 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3337999999999995e-05 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502667+0000", 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6468e-05 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116555 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5349376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:30.879 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.370114986999994, 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034965729000000001, 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116555 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:30.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:30.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:30.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1899999999999998e-07 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467908+0000", 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5740000000000001e-06 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467914+0000", 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9939999999999997e-06 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467918+0000", 2024-09-12T07:21:30.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.622e-06 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502696+0000", 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034777798999999998 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502780+0000", 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3551e-05 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502872+0000", 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.187e-05 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116557 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3506176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.369020573, 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034335510999999999, 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.884 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.885 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.885 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.885 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.885 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116557 2024-09-12T07:21:30.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.885 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:30.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:30.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469001+0000", 2024-09-12T07:21:30.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469003+0000", 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469012+0000", 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8030000000000003e-06 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469019+0000", 2024-09-12T07:21:30.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6959999999999999e-06 2024-09-12T07:21:30.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503153+0000", 2024-09-12T07:21:30.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034134096000000003 2024-09-12T07:21:30.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503236+0000", 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2886000000000006e-05 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503336+0000", 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9919000000000003e-05 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116558 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3469312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.889 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:30.890 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.368990292000007, 2024-09-12T07:21:30.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034519743999999998, 2024-09-12T07:21:30.890 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116558 2024-09-12T07:21:30.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.891 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:30.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:30.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:30.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:30.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469034+0000", 2024-09-12T07:21:30.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:21:30.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469042+0000", 2024-09-12T07:21:30.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-12T07:21:30.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469047+0000", 2024-09-12T07:21:30.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:21:30.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503370+0000", 2024-09-12T07:21:30.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034323201999999997 2024-09-12T07:21:30.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503454+0000", 2024-09-12T07:21:30.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4109000000000004e-05 2024-09-12T07:21:30.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503550+0000", 2024-09-12T07:21:30.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6009999999999997e-05 2024-09-12T07:21:30.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.895 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.895 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116559 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3551232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.895 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:30.895 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.368963460000003, 2024-09-12T07:21:30.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034689659999999997, 2024-09-12T07:21:30.896 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116559 2024-09-12T07:21:30.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.896 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469058+0000", 2024-09-12T07:21:30.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469061+0000", 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.497e-06 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469069+0000", 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2579999999999995e-06 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469074+0000", 2024-09-12T07:21:30.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1279999999999999e-06 2024-09-12T07:21:30.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503582+0000", 2024-09-12T07:21:30.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034508351 2024-09-12T07:21:30.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503656+0000", 2024-09-12T07:21:30.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3234999999999999e-05 2024-09-12T07:21:30.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503747+0000", 2024-09-12T07:21:30.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1638999999999995e-05 2024-09-12T07:21:30.900 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.900 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.900 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116560 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.368936731999995, 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034902912000000001, 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.901 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116560 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469085+0000", 2024-09-12T07:21:30.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4700000000000001e-07 2024-09-12T07:21:30.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469087+0000", 2024-09-12T07:21:30.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4760000000000001e-06 2024-09-12T07:21:30.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469095+0000", 2024-09-12T07:21:30.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2339999999999994e-06 2024-09-12T07:21:30.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469100+0000", 2024-09-12T07:21:30.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-12T07:21:30.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503796+0000", 2024-09-12T07:21:30.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034695411000000002 2024-09-12T07:21:30.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503868+0000", 2024-09-12T07:21:30.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2032000000000004e-05 2024-09-12T07:21:30.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503987+0000", 2024-09-12T07:21:30.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119369 2024-09-12T07:21:30.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.905 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.906 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116561 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3592192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.906 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:30.906 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.368913332999995, 2024-09-12T07:21:30.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035102170000000002, 2024-09-12T07:21:30.906 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116561 2024-09-12T07:21:30.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.908 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:30.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:30.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:30.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:21:30.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469110+0000", 2024-09-12T07:21:30.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1629999999999999e-06 2024-09-12T07:21:30.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469116+0000", 2024-09-12T07:21:30.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.251e-06 2024-09-12T07:21:30.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469121+0000", 2024-09-12T07:21:30.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8210000000000001e-06 2024-09-12T07:21:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504017+0000", 2024-09-12T07:21:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034895772999999998 2024-09-12T07:21:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504108+0000", 2024-09-12T07:21:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1027999999999997e-05 2024-09-12T07:21:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504210+0000", 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101729 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116562 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.368892489000004, 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035285642999999998, 2024-09-12T07:21:30.912 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116562 2024-09-12T07:21:30.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.913 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:30.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:30.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469129+0000", 2024-09-12T07:21:30.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469131+0000", 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1550000000000001e-06 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469138+0000", 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5520000000000001e-06 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469142+0000", 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9099999999999996e-06 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504239+0000", 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.03509669 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504311+0000", 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1650999999999996e-05 2024-09-12T07:21:30.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504414+0000", 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103286 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116563 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.368871855999998, 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035471691, 2024-09-12T07:21:30.917 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116563 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:30.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:30.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:21:30.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469153+0000", 2024-09-12T07:21:30.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.05e-06 2024-09-12T07:21:30.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469158+0000", 2024-09-12T07:21:30.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.874e-06 2024-09-12T07:21:30.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469163+0000", 2024-09-12T07:21:30.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8069999999999999e-06 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504444+0000", 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035280793999999997 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504523+0000", 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9036999999999996e-05 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504621+0000", 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7743000000000003e-05 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116564 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5435392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.368850745000003, 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035734657000000003, 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116564 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469171+0000", 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469172+0000", 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.832e-06 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469179+0000", 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1449999999999996e-06 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469184+0000", 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504721+0000", 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035536890000000002 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504809+0000", 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7229000000000001e-05 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504905+0000", 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6337999999999998e-05 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116565 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5476352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:30.926 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.368830079000006, 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035933158, 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116565 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469193+0000", 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.973e-06 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469201+0000", 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3730000000000004e-06 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469206+0000", 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1660000000000002e-06 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504937+0000", 2024-09-12T07:21:30.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035731205000000002 2024-09-12T07:21:30.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505035+0000", 2024-09-12T07:21:30.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7759999999999999e-05 2024-09-12T07:21:30.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505124+0000", 2024-09-12T07:21:30.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9258000000000005e-05 2024-09-12T07:21:30.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.931 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.931 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116566 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5271552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.931 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:30.931 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.368807493000006, 2024-09-12T07:21:30.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036116848999999999, 2024-09-12T07:21:30.931 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.932 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116566 2024-09-12T07:21:30.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.932 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:30.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:30.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469214+0000", 2024-09-12T07:21:30.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:30.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469216+0000", 2024-09-12T07:21:30.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8759999999999999e-06 2024-09-12T07:21:30.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469222+0000", 2024-09-12T07:21:30.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.173e-06 2024-09-12T07:21:30.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469227+0000", 2024-09-12T07:21:30.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3070000000000002e-06 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505154+0000", 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035926362000000003 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505239+0000", 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5261999999999998e-05 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505330+0000", 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1436999999999999e-05 2024-09-12T07:21:30.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116567 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5312512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.368786963999995, 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036305875000000001, 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116567 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.58e-07 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469242+0000", 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469249+0000", 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5549999999999999e-06 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469254+0000", 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9899999999999997e-06 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505361+0000", 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036106923999999999 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505449+0000", 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8531999999999999e-05 2024-09-12T07:21:30.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505540+0000", 2024-09-12T07:21:30.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0805999999999998e-05 2024-09-12T07:21:30.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.940 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.940 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116568 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5353472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.368760291000001, 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036561383000000003, 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116568 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469263+0000", 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7549999999999999e-06 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469269+0000", 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7219999999999996e-06 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469273+0000", 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:30.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505631+0000", 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036357997000000003 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505711+0000", 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9331000000000003e-05 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505822+0000", 2024-09-12T07:21:30.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000111208 2024-09-12T07:21:30.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.945 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:30.945 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:30.946 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:21:30.947 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:30.948+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:31.047 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:31.049+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:31.095 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:31.095 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:31.103 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:21:31.148 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:31.150+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:31.159 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T07:21:31.229 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:31.230 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:31.239 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:21:31.283 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:31.285+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:31.350 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:31.352+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:31.354 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77440 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13156352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:21:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.109912945999994, 2024-09-12T07:21:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051399724000000001, 2024-09-12T07:21:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77440 2024-09-12T07:21:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:21:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.245444+0000", 2024-09-12T07:21:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246267+0000", 2024-09-12T07:21:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082256199999999999 2024-09-12T07:21:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246271+0000", 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6700000000000002e-06 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246283+0000", 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.207e-05 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246291+0000", 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7090000000000002e-06 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296645+0000", 2024-09-12T07:21:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.050353872000000001 2024-09-12T07:21:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296729+0000", 2024-09-12T07:21:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4215000000000006e-05 2024-09-12T07:21:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.296844+0000", 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114626 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77446 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.108914431000002, 2024-09-12T07:21:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051543643, 2024-09-12T07:21:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77446 2024-09-12T07:21:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:21:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246443+0000", 2024-09-12T07:21:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246579+0000", 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013651399999999999 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246583+0000", 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1509999999999997e-06 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246593+0000", 2024-09-12T07:21:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5670000000000001e-06 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246599+0000", 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5189999999999998e-06 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297826+0000", 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051226381000000001 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297903+0000", 2024-09-12T07:21:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6710000000000002e-05 2024-09-12T07:21:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.297986+0000", 2024-09-12T07:21:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3801000000000006e-05 2024-09-12T07:21:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77447 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 5783552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.108746816999997, 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051512637, 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77447 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246610+0000", 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246812+0000", 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000201506 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246817+0000", 2024-09-12T07:21:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:21:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246828+0000", 2024-09-12T07:21:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1693e-05 2024-09-12T07:21:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246836+0000", 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4429999999999997e-06 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298021+0000", 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051184940999999998 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298067+0000", 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6292000000000003e-05 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298123+0000", 2024-09-12T07:21:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5828999999999997e-05 2024-09-12T07:21:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77448 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:21:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.108507079000006, 2024-09-12T07:21:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051506024999999997, 2024-09-12T07:21:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77448 2024-09-12T07:21:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246850+0000", 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246892+0000", 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1854999999999998e-05 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246895+0000", 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3129999999999999e-06 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246904+0000", 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6179999999999995e-06 2024-09-12T07:21:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246911+0000", 2024-09-12T07:21:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7939999999999997e-06 2024-09-12T07:21:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298164+0000", 2024-09-12T07:21:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051253559999999997 2024-09-12T07:21:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298249+0000", 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4549000000000007e-05 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298356+0000", 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107336 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77449 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:21:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.108435882999999, 2024-09-12T07:21:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051582830000000003, 2024-09-12T07:21:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77449 2024-09-12T07:21:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:21:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.246921+0000", 2024-09-12T07:21:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247137+0000", 2024-09-12T07:21:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021599499999999999 2024-09-12T07:21:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247142+0000", 2024-09-12T07:21:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:21:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247154+0000", 2024-09-12T07:21:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1878e-05 2024-09-12T07:21:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247168+0000", 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4114000000000001e-05 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298391+0000", 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051223249999999998 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298446+0000", 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4366000000000001e-05 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298504+0000", 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8273000000000003e-05 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77450 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12681216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.108176313000001, 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051562375000000001, 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77450 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247181+0000", 2024-09-12T07:21:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2799999999999996e-07 2024-09-12T07:21:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247184+0000", 2024-09-12T07:21:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7369999999999998e-06 2024-09-12T07:21:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247192+0000", 2024-09-12T07:21:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5920000000000003e-06 2024-09-12T07:21:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247197+0000", 2024-09-12T07:21:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6720000000000001e-06 2024-09-12T07:21:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298550+0000", 2024-09-12T07:21:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051352082 2024-09-12T07:21:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298641+0000", 2024-09-12T07:21:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1236000000000005e-05 2024-09-12T07:21:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298743+0000", 2024-09-12T07:21:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102428 2024-09-12T07:21:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77451 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7061504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:21:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.108152199000003, 2024-09-12T07:21:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051813773, 2024-09-12T07:21:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77451 2024-09-12T07:21:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:21:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247205+0000", 2024-09-12T07:21:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247335+0000", 2024-09-12T07:21:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013012599999999999 2024-09-12T07:21:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247340+0000", 2024-09-12T07:21:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.544e-06 2024-09-12T07:21:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247351+0000", 2024-09-12T07:21:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.111e-05 2024-09-12T07:21:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247358+0000", 2024-09-12T07:21:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9859999999999997e-06 2024-09-12T07:21:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298909+0000", 2024-09-12T07:21:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051551781999999997 2024-09-12T07:21:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.298960+0000", 2024-09-12T07:21:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0594999999999999e-05 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299019+0000", 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8629999999999999e-05 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77452 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.107987418999997, 2024-09-12T07:21:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051892134999999999, 2024-09-12T07:21:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77452 2024-09-12T07:21:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247370+0000", 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247535+0000", 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165508 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247552+0000", 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6514000000000001e-05 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247565+0000", 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3372e-05 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247577+0000", 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2143999999999999e-05 2024-09-12T07:21:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299064+0000", 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051486697999999997 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299150+0000", 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6549000000000001e-05 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299262+0000", 2024-09-12T07:21:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011135 2024-09-12T07:21:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77453 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7553024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:21:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.107766698000006, 2024-09-12T07:21:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051901623000000001, 2024-09-12T07:21:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77453 2024-09-12T07:21:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:21:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247590+0000", 2024-09-12T07:21:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247710+0000", 2024-09-12T07:21:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011951 2024-09-12T07:21:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247713+0000", 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4539999999999999e-06 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247720+0000", 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0339999999999999e-06 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247730+0000", 2024-09-12T07:21:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0030000000000001e-05 2024-09-12T07:21:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299297+0000", 2024-09-12T07:21:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051566446000000002 2024-09-12T07:21:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299382+0000", 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4963000000000006e-05 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299492+0000", 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011018600000000001 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77454 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7618560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.107618150999997, 2024-09-12T07:21:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051921851999999998, 2024-09-12T07:21:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77454 2024-09-12T07:21:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:21:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247739+0000", 2024-09-12T07:21:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247900+0000", 2024-09-12T07:21:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016088699999999999 2024-09-12T07:21:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247904+0000", 2024-09-12T07:21:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6950000000000002e-06 2024-09-12T07:21:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247912+0000", 2024-09-12T07:21:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7129999999999994e-06 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247921+0000", 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0299999999999999e-06 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299526+0000", 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051604744000000001 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299601+0000", 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5129000000000005e-05 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299661+0000", 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9654000000000002e-05 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77455 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12746752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:21:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.107424283, 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051971254000000001, 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77455 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.247933+0000", 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248181+0000", 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024790000000000001 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248185+0000", 2024-09-12T07:21:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5079999999999999e-06 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248197+0000", 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.132e-05 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248203+0000", 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6320000000000002e-06 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299705+0000", 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051501342999999998 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299799+0000", 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4340999999999997e-05 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299904+0000", 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010521 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77456 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 7979008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.107142057999994, 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051860577999999997, 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77456 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248215+0000", 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248545+0000", 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033009700000000001 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248550+0000", 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248560+0000", 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:21:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248568+0000", 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4939999999999997e-06 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.299973+0000", 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051405470000000002 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300019+0000", 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5448999999999998e-05 2024-09-12T07:21:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300076+0000", 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7073000000000001e-05 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77457 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13172736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.106775729999995, 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051737914000000003, 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77457 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248581+0000", 2024-09-12T07:21:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248582+0000", 2024-09-12T07:21:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0100000000000005e-07 2024-09-12T07:21:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248584+0000", 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248593+0000", 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9800000000000004e-06 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248609+0000", 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5664000000000001e-05 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300117+0000", 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051507605999999997 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300212+0000", 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4919000000000004e-05 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300319+0000", 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010776399999999999 2024-09-12T07:21:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77458 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.106733312000003, 2024-09-12T07:21:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051863578, 2024-09-12T07:21:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77458 2024-09-12T07:21:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248624+0000", 2024-09-12T07:21:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3399999999999999e-07 2024-09-12T07:21:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248627+0000", 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1389999999999999e-06 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248635+0000", 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4019999999999996e-06 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248641+0000", 2024-09-12T07:21:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1689999999999997e-06 2024-09-12T07:21:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300359+0000", 2024-09-12T07:21:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051717889000000003 2024-09-12T07:21:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300418+0000", 2024-09-12T07:21:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8689e-05 2024-09-12T07:21:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300487+0000", 2024-09-12T07:21:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9756000000000001e-05 2024-09-12T07:21:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77459 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 13238272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:21:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.106707619000005, 2024-09-12T07:21:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052076504000000003, 2024-09-12T07:21:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77459 2024-09-12T07:21:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:21:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.248649+0000", 2024-09-12T07:21:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250611+0000", 2024-09-12T07:21:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019617879999999999 2024-09-12T07:21:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250618+0000", 2024-09-12T07:21:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1829999999999997e-06 2024-09-12T07:21:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250629+0000", 2024-09-12T07:21:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0438000000000001e-05 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250648+0000", 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9355e-05 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300528+0000", 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.049879252999999998 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300619+0000", 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1910000000000006e-05 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.300726+0000", 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010657700000000001 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77467 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.104501799999994, 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05160754, 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77467 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:21:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250855+0000", 2024-09-12T07:21:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250856+0000", 2024-09-12T07:21:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6800000000000001e-07 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250858+0000", 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9180000000000001e-06 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250864+0000", 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.066e-06 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250869+0000", 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302283+0000", 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051414591000000003 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302378+0000", 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4283000000000005e-05 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302463+0000", 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5249999999999999e-05 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77468 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14909440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.104482227000005, 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051742162000000001, 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77468 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250875+0000", 2024-09-12T07:21:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2399999999999999e-07 2024-09-12T07:21:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250877+0000", 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.815e-06 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250883+0000", 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250888+0000", 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.933e-06 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302496+0000", 2024-09-12T07:21:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051607394000000001 2024-09-12T07:21:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302553+0000", 2024-09-12T07:21:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6971000000000001e-05 2024-09-12T07:21:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302617+0000", 2024-09-12T07:21:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4418999999999994e-05 2024-09-12T07:21:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77469 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 14974976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.104462048000002, 2024-09-12T07:21:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965694, 2024-09-12T07:21:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77469 2024-09-12T07:21:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250895+0000", 2024-09-12T07:21:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3300000000000003e-07 2024-09-12T07:21:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250897+0000", 2024-09-12T07:21:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9130000000000001e-06 2024-09-12T07:21:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250903+0000", 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0129999999999999e-06 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250908+0000", 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9679999999999997e-06 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302661+0000", 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051753066 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302755+0000", 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3376999999999997e-05 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302861+0000", 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105924 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77470 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12828672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.104442371000005, 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052098589000000001, 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77470 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250915+0000", 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250917+0000", 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8190000000000001e-06 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250923+0000", 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1800000000000001e-06 2024-09-12T07:21:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250928+0000", 2024-09-12T07:21:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9420000000000003e-06 2024-09-12T07:21:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302894+0000", 2024-09-12T07:21:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.051965863000000001 2024-09-12T07:21:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.302955+0000", 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1235999999999994e-05 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303013+0000", 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8167000000000001e-05 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:77471 3.9 3:9513217c:::rbd_data.116063d335b3.0000000000000005:8 [read 12894208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-12T07:21:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:21:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.104422825, 2024-09-12T07:21:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.052326263999999997, 2024-09-12T07:21:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 77471 2024-09-12T07:21:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:21:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.250934+0000", 2024-09-12T07:21:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256961+0000", 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060271209999999999 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256972+0000", 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0755e-05 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256986+0000", 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3845e-05 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.256996+0000", 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9809999999999998e-06 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303060+0000", 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.046063661999999998 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303149+0000", 2024-09-12T07:21:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8949999999999994e-05 2024-09-12T07:21:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:04.303261+0000", 2024-09-12T07:21:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011195 2024-09-12T07:21:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.469 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:31.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:31.471 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:31.472+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:31.571 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:31.572+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:31.619 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:31.619 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:31.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:31.671 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:31.673+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:31.772 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:31.774+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:31.773 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:31.774 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:31.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:31.873 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:31.874+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:31.892 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169456 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1527808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.372657445000002, 2024-09-12T07:21:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068013471000000006, 2024-09-12T07:21:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169456 2024-09-12T07:21:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520228+0000", 2024-09-12T07:21:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.63e-07 2024-09-12T07:21:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520230+0000", 2024-09-12T07:21:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3420000000000002e-06 2024-09-12T07:21:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520237+0000", 2024-09-12T07:21:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5950000000000004e-06 2024-09-12T07:21:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520242+0000", 2024-09-12T07:21:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588003+0000", 2024-09-12T07:21:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067761181000000004 2024-09-12T07:21:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588071+0000", 2024-09-12T07:21:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8095000000000004e-05 2024-09-12T07:21:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588241+0000", 2024-09-12T07:21:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016951999999999999 2024-09-12T07:21:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169457 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1568768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.37263566, 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394231, 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169457 2024-09-12T07:21:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520250+0000", 2024-09-12T07:21:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6899999999999997e-07 2024-09-12T07:21:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520252+0000", 2024-09-12T07:21:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1959999999999998e-06 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520259+0000", 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1509999999999998e-06 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520264+0000", 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588275+0000", 2024-09-12T07:21:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068010854999999995 2024-09-12T07:21:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588350+0000", 2024-09-12T07:21:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5402999999999996e-05 2024-09-12T07:21:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588643+0000", 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029306599999999998 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169458 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.372613183999999, 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069257746999999995, 2024-09-12T07:21:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169458 2024-09-12T07:21:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520274+0000", 2024-09-12T07:21:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3350000000000001e-06 2024-09-12T07:21:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520281+0000", 2024-09-12T07:21:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6819999999999997e-06 2024-09-12T07:21:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520286+0000", 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0479999999999998e-06 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588740+0000", 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068453571000000005 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588810+0000", 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0611000000000005e-05 2024-09-12T07:21:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589529+0000", 2024-09-12T07:21:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071908599999999997 2024-09-12T07:21:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169459 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1650688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.372590892000002, 2024-09-12T07:21:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069503178999999998, 2024-09-12T07:21:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169459 2024-09-12T07:21:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:21:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520296+0000", 2024-09-12T07:21:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1890000000000001e-06 2024-09-12T07:21:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520303+0000", 2024-09-12T07:21:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6749999999999996e-06 2024-09-12T07:21:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520308+0000", 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1240000000000004e-06 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589563+0000", 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069254688999999994 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589637+0000", 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3711e-05 2024-09-12T07:21:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589797+0000", 2024-09-12T07:21:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016039799999999999 2024-09-12T07:21:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169460 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1691648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.372569840000001, 2024-09-12T07:21:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069750706999999995, 2024-09-12T07:21:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169460 2024-09-12T07:21:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7399999999999999e-07 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520318+0000", 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2639999999999998e-06 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520324+0000", 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3260000000000001e-06 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520329+0000", 2024-09-12T07:21:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6959999999999996e-06 2024-09-12T07:21:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589830+0000", 2024-09-12T07:21:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069501010000000002 2024-09-12T07:21:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589903+0000", 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3558000000000003e-05 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590066+0000", 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016247899999999999 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169461 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.372548712, 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069982339000000005, 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169461 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520339+0000", 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520345+0000", 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5359999999999998e-06 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520350+0000", 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.079e-06 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590098+0000", 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069747117999999997 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590168+0000", 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9982999999999997e-05 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590318+0000", 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015076799999999999 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169462 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1773568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.372526597, 2024-09-12T07:21:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070210122, 2024-09-12T07:21:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169462 2024-09-12T07:21:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520359+0000", 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520361+0000", 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0109999999999999e-06 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520367+0000", 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.285e-06 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520374+0000", 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0899999999999999e-06 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590348+0000", 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069974233999999996 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590418+0000", 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9314999999999996e-05 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590568+0000", 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000150693 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169463 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.372502703999999, 2024-09-12T07:21:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070436171000000006, 2024-09-12T07:21:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169463 2024-09-12T07:21:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524036+0000", 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036543130000000002 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524043+0000", 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0639999999999996e-06 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524058+0000", 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.411e-05 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524065+0000", 2024-09-12T07:21:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9780000000000005e-06 2024-09-12T07:21:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590598+0000", 2024-09-12T07:21:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066532540000000001 2024-09-12T07:21:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590669+0000", 2024-09-12T07:21:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0598000000000004e-05 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590818+0000", 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014956799999999999 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169469 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.36868544, 2024-09-12T07:21:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068176724999999994, 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169469 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524200+0000", 2024-09-12T07:21:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524202+0000", 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5100000000000001e-06 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524209+0000", 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524213+0000", 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592153+0000", 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067939948 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592225+0000", 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2078999999999999e-05 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592376+0000", 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015063000000000001 2024-09-12T07:21:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169470 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2101248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.368664419000002, 2024-09-12T07:21:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394620000000003, 2024-09-12T07:21:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169470 2024-09-12T07:21:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524221+0000", 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8700000000000001e-07 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524223+0000", 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1619999999999998e-06 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524230+0000", 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.657e-06 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524235+0000", 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0799999999999996e-06 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592429+0000", 2024-09-12T07:21:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068194553000000005 2024-09-12T07:21:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592512+0000", 2024-09-12T07:21:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3123999999999999e-05 2024-09-12T07:21:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592615+0000", 2024-09-12T07:21:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102657 2024-09-12T07:21:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169471 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2142208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.368643634000001, 2024-09-12T07:21:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068565151000000005, 2024-09-12T07:21:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169471 2024-09-12T07:21:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6300000000000001e-07 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524244+0000", 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524250+0000", 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1890000000000004e-06 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524256+0000", 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5230000000000003e-06 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592647+0000", 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068391788999999995 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592719+0000", 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1859000000000004e-05 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592806+0000", 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6940000000000004e-05 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169472 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.368622469999998, 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068745155000000002, 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169472 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524263+0000", 2024-09-12T07:21:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4499999999999998e-07 2024-09-12T07:21:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524265+0000", 2024-09-12T07:21:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.176e-06 2024-09-12T07:21:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524271+0000", 2024-09-12T07:21:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4439999999999996e-06 2024-09-12T07:21:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524276+0000", 2024-09-12T07:21:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7949999999999998e-06 2024-09-12T07:21:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592836+0000", 2024-09-12T07:21:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068560461000000003 2024-09-12T07:21:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592907+0000", 2024-09-12T07:21:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0258000000000004e-05 2024-09-12T07:21:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593007+0000", 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100676 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169473 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2224128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.3686021, 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068911932999999995, 2024-09-12T07:21:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169473 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.34e-07 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524285+0000", 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0209999999999998e-06 2024-09-12T07:21:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524291+0000", 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8989999999999998e-06 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524296+0000", 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0069999999999997e-06 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593038+0000", 2024-09-12T07:21:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068742019000000001 2024-09-12T07:21:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593109+0000", 2024-09-12T07:21:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1075999999999995e-05 2024-09-12T07:21:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593195+0000", 2024-09-12T07:21:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5576999999999997e-05 2024-09-12T07:21:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169474 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2265088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.368581905999999, 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069079421000000002, 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169474 2024-09-12T07:21:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3700000000000001e-07 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524305+0000", 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1579999999999999e-06 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524311+0000", 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524316+0000", 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1109999999999999e-06 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593225+0000", 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068909140999999993 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593298+0000", 2024-09-12T07:21:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2432e-05 2024-09-12T07:21:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593382+0000", 2024-09-12T07:21:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4404e-05 2024-09-12T07:21:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169475 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2306048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.368561995, 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069270256000000002, 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169475 2024-09-12T07:21:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7500000000000001e-07 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524325+0000", 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1509999999999998e-06 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524332+0000", 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2650000000000002e-06 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524336+0000", 2024-09-12T07:21:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:21:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593418+0000", 2024-09-12T07:21:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069081988999999996 2024-09-12T07:21:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593491+0000", 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2619000000000004e-05 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593593+0000", 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101887 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169476 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.368537915000001, 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069449397999999996, 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169476 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4900000000000001e-07 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524350+0000", 2024-09-12T07:21:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.481e-06 2024-09-12T07:21:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524356+0000", 2024-09-12T07:21:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6989999999999997e-06 2024-09-12T07:21:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524362+0000", 2024-09-12T07:21:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2320000000000001e-06 2024-09-12T07:21:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593623+0000", 2024-09-12T07:21:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069261328999999996 2024-09-12T07:21:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593706+0000", 2024-09-12T07:21:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2528999999999995e-05 2024-09-12T07:21:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593796+0000", 2024-09-12T07:21:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0679000000000004e-05 2024-09-12T07:21:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169477 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2387968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.368515110000001, 2024-09-12T07:21:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069660978999999998, 2024-09-12T07:21:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169477 2024-09-12T07:21:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3299999999999998e-07 2024-09-12T07:21:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524372+0000", 2024-09-12T07:21:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1519999999999999e-06 2024-09-12T07:21:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524379+0000", 2024-09-12T07:21:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.562e-06 2024-09-12T07:21:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524383+0000", 2024-09-12T07:21:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5430000000000004e-06 2024-09-12T07:21:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593852+0000", 2024-09-12T07:21:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069468641999999997 2024-09-12T07:21:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593926+0000", 2024-09-12T07:21:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4070000000000001e-05 2024-09-12T07:21:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594031+0000", 2024-09-12T07:21:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104677 2024-09-12T07:21:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169478 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2428928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.368494693999999, 2024-09-12T07:21:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069830325999999998, 2024-09-12T07:21:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169478 2024-09-12T07:21:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524393+0000", 2024-09-12T07:21:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.092e-06 2024-09-12T07:21:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524399+0000", 2024-09-12T07:21:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3539999999999996e-06 2024-09-12T07:21:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524404+0000", 2024-09-12T07:21:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9590000000000003e-06 2024-09-12T07:21:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594062+0000", 2024-09-12T07:21:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069658017000000003 2024-09-12T07:21:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594131+0000", 2024-09-12T07:21:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9226000000000004e-05 2024-09-12T07:21:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594220+0000", 2024-09-12T07:21:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9295999999999993e-05 2024-09-12T07:21:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169479 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2469888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.36847422, 2024-09-12T07:21:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069996201999999993, 2024-09-12T07:21:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169479 2024-09-12T07:21:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530070+0000", 2024-09-12T07:21:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056590099999999999 2024-09-12T07:21:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530075+0000", 2024-09-12T07:21:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8200000000000002e-06 2024-09-12T07:21:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530085+0000", 2024-09-12T07:21:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-12T07:21:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530093+0000", 2024-09-12T07:21:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5519999999999999e-06 2024-09-12T07:21:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594250+0000", 2024-09-12T07:21:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064157642000000001 2024-09-12T07:21:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594318+0000", 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.737e-05 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594407+0000", 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9117999999999996e-05 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169502 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 3411968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.362019295, 2024-09-12T07:21:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068036163999999996, 2024-09-12T07:21:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169502 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535987+0000", 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051212549999999999 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535993+0000", 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0639999999999999e-06 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536003+0000", 2024-09-12T07:21:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0549e-05 2024-09-12T07:21:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536012+0000", 2024-09-12T07:21:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2309999999999996e-06 2024-09-12T07:21:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598753+0000", 2024-09-12T07:21:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062741665000000002 2024-09-12T07:21:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598822+0000", 2024-09-12T07:21:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8689999999999995e-05 2024-09-12T07:21:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598902+0000", 2024-09-12T07:21:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9709999999999994e-05 2024-09-12T07:21:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:32.003 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:32.004 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:32.005 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:32.006+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:32.104 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:32.106+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:32.125 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:32.125 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:32.125 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:32.125 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:32.135 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:32.147 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is set 2024-09-12T07:21:32.206 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:32.207+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:32.248 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:32.248 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:32.249 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:32.249 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:32.257 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:32.306 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:32.308+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:32.406 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 94.793689732999994, 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:21:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 89.658229477000006, 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:21:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 89.654002356000007, 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:21:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:21:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:21:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:21:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:21:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 88.926251700999998, 2024-09-12T07:21:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:21:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:21:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 88.837275112, 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:21:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:21:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 88.805723782000001, 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:21:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 88.802523246000007, 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:21:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:21:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 88.799277497999995, 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:21:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 88.797035667000003, 2024-09-12T07:21:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:21:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 88.705928463000006, 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:21:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:21:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:21:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:21:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:21:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:21:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:21:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 88.702100375000001, 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:21:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 65.720187593999995, 2024-09-12T07:21:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:21:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:21:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:21:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 65.708373949999995, 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:21:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:21:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:21:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:21:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:21:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:21:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:21:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:21:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:21:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:21:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 65.707037903, 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:21:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165942 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 24.206203889000001, 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93685439599999998, 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165942 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200798+0000", 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.455e-06 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200802+0000", 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2390000000000004e-06 2024-09-12T07:21:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200814+0000", 2024-09-12T07:21:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1816e-05 2024-09-12T07:21:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200820+0000", 2024-09-12T07:21:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8760000000000002e-06 2024-09-12T07:21:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200899+0000", 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8695999999999993e-05 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200912+0000", 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3545e-05 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137326+0000", 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93641327100000005 2024-09-12T07:21:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137519+0000", 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000193298 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137651+0000", 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013219999999999999 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165957 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 24.205592012, 2024-09-12T07:21:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93691399799999997, 2024-09-12T07:21:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165957 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201409+0000", 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1999999999999999e-07 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201412+0000", 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2590000000000001e-06 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201423+0000", 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0547000000000001e-05 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201428+0000", 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.101e-06 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201488+0000", 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9426999999999999e-05 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201498+0000", 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0883999999999999e-05 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138057+0000", 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93655902000000002 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138168+0000", 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011034199999999999 2024-09-12T07:21:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138322+0000", 2024-09-12T07:21:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015469799999999999 2024-09-12T07:21:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165958 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 24.205560940000002, 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.937070242, 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165958 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.58e-07 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201442+0000", 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.719e-06 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201447+0000", 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4770000000000002e-06 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201450+0000", 2024-09-12T07:21:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.219e-06 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201530+0000", 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9833000000000006e-05 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201538+0000", 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.028e-06 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138367+0000", 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93682842899999996 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138429+0000", 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1890000000000005e-05 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138510+0000", 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1289e-05 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165973 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 24.205428314999999, 2024-09-12T07:21:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93711003900000001, 2024-09-12T07:21:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165973 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201573+0000", 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8599999999999996e-07 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201580+0000", 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3810000000000004e-06 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201590+0000", 2024-09-12T07:21:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0169999999999999e-05 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201596+0000", 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4110000000000001e-06 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201645+0000", 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9156999999999997e-05 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201656+0000", 2024-09-12T07:21:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1112e-05 2024-09-12T07:21:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138538+0000", 2024-09-12T07:21:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93688176899999998 2024-09-12T07:21:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138595+0000", 2024-09-12T07:21:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6759000000000003e-05 2024-09-12T07:21:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138682+0000", 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7293999999999995e-05 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165974 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 24.205390167000001, 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93725339100000005, 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165974 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201611+0000", 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201614+0000", 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.757e-06 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201620+0000", 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.015e-06 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201623+0000", 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6210000000000001e-06 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201679+0000", 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6017999999999998e-05 2024-09-12T07:21:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201685+0000", 2024-09-12T07:21:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2560000000000002e-06 2024-09-12T07:21:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138699+0000", 2024-09-12T07:21:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93701424600000005 2024-09-12T07:21:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138749+0000", 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9951000000000003e-05 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138864+0000", 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115017 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-12T07:21:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 9.9114702060000006, 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0249796689999999, 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495532+0000", 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6130000000000001e-06 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495542+0000", 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7990000000000005e-06 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495556+0000", 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4737e-05 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495567+0000", 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0348e-05 2024-09-12T07:21:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495628+0000", 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0711999999999997e-05 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495638+0000", 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0482999999999999e-05 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520066+0000", 2024-09-12T07:21:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0244278959999999 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520246+0000", 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000179815 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520510+0000", 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026426599999999998 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:32.504 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:32.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:32.506 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:32.506+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:32.606 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:32.607+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:32.613 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:32.613 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:32.622 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:32.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:32.707 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:32.708+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:32.768 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:32.768 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:32.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:32.791 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:32.792 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:32.808 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:32.809+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:32.809 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:32.809 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:32.825 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:32.825 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:32.841 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:32.841 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:32.858 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:32.875 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:32.875 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:32.884 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:32.884 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:32.891 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:32.893 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:32.908 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:32.908 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:32.908 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:32.910+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:33.003 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:33.004 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:33.009 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:33.010+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:33.012 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:33.110 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:33.111+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:33.118 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:33.119 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:33.127 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:33.151 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 6, 7, 3] out_osds: [1, 5, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:21:33.151 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:21:33.151 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-12T07:21:33.151 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-12T07:21:33.151 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-12T07:21:33.152 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-12T07:21:33.211 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:33.213+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:33.241 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:33.241 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:33.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:33.271 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:33.271 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-12T07:21:33.271 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:33.312 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:33.314+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:33.398 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:33.398 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:33.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:33.452 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:33.454+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:33.513 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:33.514+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:33.515 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:33.515 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:33.524 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:33.614 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:33.615+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:33.672 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:33.672 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:33.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:33.724 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:33.726+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:33.793 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:33.793 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:33.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:33.816 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:33.818+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:33.916 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:33.918+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:33.949 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:33.949 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:33.957 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:34.041 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:34.043+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:34.105 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:34.105 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:34.113 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:34.118 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:34.120+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:34.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:21:34.241 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:34.243+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:34.281 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:34.281 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:34.291 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:34.320 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:34.322+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:34.421 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:34.423+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:34.440 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:34.440 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:34.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:34.522 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:34.524+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:34.556 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:34.556 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:34.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:21:34.649 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:34.650+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:34.712 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:34.712 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:34.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:21:34.762 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:34.763+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:34.824 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:34.826+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:34.868 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:34.868 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:34.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:21:34.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:34.925 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:34.927+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:35.026 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:35.027+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:35.027 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:35.027 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:35.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:35.044 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:35.061 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:35.061 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:35.078 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:35.094 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:35.112 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:35.112 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:35.127 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:35.128+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:35.129 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:21:35.152 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:35.152 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:35.165 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:35.165 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:35.165 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:35.165 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:35.176 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:35.178 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:35.227 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:35.229+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:35.295 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:35.295 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:35.295 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:35.295 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:35.295 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:35.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:35.328 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:35.329+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:35.412 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169456 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1527808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.893307819, 2024-09-12T07:21:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068013471000000006, 2024-09-12T07:21:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169456 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520228+0000", 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.63e-07 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520230+0000", 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3420000000000002e-06 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520237+0000", 2024-09-12T07:21:35.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5950000000000004e-06 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520242+0000", 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588003+0000", 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067761181000000004 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588071+0000", 2024-09-12T07:21:35.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8095000000000004e-05 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588241+0000", 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016951999999999999 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169457 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1568768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.893286033999999, 2024-09-12T07:21:35.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394231, 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169457 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520250+0000", 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6899999999999997e-07 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520252+0000", 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1959999999999998e-06 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520259+0000", 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1509999999999998e-06 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520264+0000", 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588275+0000", 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068010854999999995 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588350+0000", 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5402999999999996e-05 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588643+0000", 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029306599999999998 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169458 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.421 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.893263558000001, 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069257746999999995, 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169458 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:35.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520274+0000", 2024-09-12T07:21:35.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3350000000000001e-06 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520281+0000", 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6819999999999997e-06 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520286+0000", 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0479999999999998e-06 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588740+0000", 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068453571000000005 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588810+0000", 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0611000000000005e-05 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589529+0000", 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071908599999999997 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169459 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1650688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.893241266, 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069503178999999998, 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169459 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520296+0000", 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1890000000000001e-06 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520303+0000", 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6749999999999996e-06 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520308+0000", 2024-09-12T07:21:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1240000000000004e-06 2024-09-12T07:21:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589563+0000", 2024-09-12T07:21:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069254688999999994 2024-09-12T07:21:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589637+0000", 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3711e-05 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589797+0000", 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016039799999999999 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169460 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1691648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:35.430 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.893220213999999, 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069750706999999995, 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169460 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7399999999999999e-07 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520318+0000", 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2639999999999998e-06 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520324+0000", 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3260000000000001e-06 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520329+0000", 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6959999999999996e-06 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589830+0000", 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069501010000000002 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589903+0000", 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3558000000000003e-05 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590066+0000", 2024-09-12T07:21:35.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016247899999999999 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169461 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.893199085999999, 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069982339000000005, 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169461 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:35.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520339+0000", 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520345+0000", 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5359999999999998e-06 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520350+0000", 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.079e-06 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590098+0000", 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069747117999999997 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590168+0000", 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9982999999999997e-05 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590318+0000", 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015076799999999999 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169462 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1773568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.439 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.893176970999999, 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070210122, 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169462 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520359+0000", 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520361+0000", 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0109999999999999e-06 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520367+0000", 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.285e-06 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520374+0000", 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0899999999999999e-06 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590348+0000", 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069974233999999996 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590418+0000", 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9314999999999996e-05 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590568+0000", 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000150693 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.443 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169463 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.893153078000001, 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070436171000000006, 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169463 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524036+0000", 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036543130000000002 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524043+0000", 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0639999999999996e-06 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524058+0000", 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.411e-05 2024-09-12T07:21:35.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524065+0000", 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9780000000000005e-06 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590598+0000", 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066532540000000001 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590669+0000", 2024-09-12T07:21:35.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0598000000000004e-05 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590818+0000", 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014956799999999999 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169469 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.889335813999999, 2024-09-12T07:21:35.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068176724999999994, 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169469 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524200+0000", 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524202+0000", 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5100000000000001e-06 2024-09-12T07:21:35.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524209+0000", 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524213+0000", 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592153+0000", 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067939948 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592225+0000", 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2078999999999999e-05 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592376+0000", 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015063000000000001 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169470 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2101248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.889314793, 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394620000000003, 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169470 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.453 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524221+0000", 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8700000000000001e-07 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524223+0000", 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1619999999999998e-06 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524230+0000", 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.657e-06 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524235+0000", 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0799999999999996e-06 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592429+0000", 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068194553000000005 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592512+0000", 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3123999999999999e-05 2024-09-12T07:21:35.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592615+0000", 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102657 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169471 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2142208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.889294008, 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068565151000000005, 2024-09-12T07:21:35.457 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169471 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6300000000000001e-07 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524244+0000", 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524250+0000", 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1890000000000004e-06 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524256+0000", 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5230000000000003e-06 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592647+0000", 2024-09-12T07:21:35.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068391788999999995 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592719+0000", 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1859000000000004e-05 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592806+0000", 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6940000000000004e-05 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169472 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.889272844000001, 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068745155000000002, 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169472 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524263+0000", 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4499999999999998e-07 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524265+0000", 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.176e-06 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524271+0000", 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4439999999999996e-06 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524276+0000", 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7949999999999998e-06 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592836+0000", 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068560461000000003 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592907+0000", 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0258000000000004e-05 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593007+0000", 2024-09-12T07:21:35.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100676 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169473 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2224128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.889252473999999, 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068911932999999995, 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.466 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169473 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.34e-07 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524285+0000", 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0209999999999998e-06 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524291+0000", 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8989999999999998e-06 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524296+0000", 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0069999999999997e-06 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593038+0000", 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068742019000000001 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593109+0000", 2024-09-12T07:21:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1075999999999995e-05 2024-09-12T07:21:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593195+0000", 2024-09-12T07:21:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5576999999999997e-05 2024-09-12T07:21:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169474 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2265088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.889232280000002, 2024-09-12T07:21:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069079421000000002, 2024-09-12T07:21:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.471 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169474 2024-09-12T07:21:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3700000000000001e-07 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524305+0000", 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1579999999999999e-06 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524311+0000", 2024-09-12T07:21:35.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524316+0000", 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1109999999999999e-06 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593225+0000", 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068909140999999993 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593298+0000", 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2432e-05 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593382+0000", 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4404e-05 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169475 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2306048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.475 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.889212368999999, 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069270256000000002, 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169475 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7500000000000001e-07 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524325+0000", 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1509999999999998e-06 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524332+0000", 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2650000000000002e-06 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524336+0000", 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593418+0000", 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069081988999999996 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593491+0000", 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2619000000000004e-05 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593593+0000", 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101887 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.479 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169476 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.889188289, 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069449397999999996, 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.480 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169476 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4900000000000001e-07 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524350+0000", 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.481e-06 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524356+0000", 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6989999999999997e-06 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524362+0000", 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2320000000000001e-06 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593623+0000", 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069261328999999996 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593706+0000", 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2528999999999995e-05 2024-09-12T07:21:35.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593796+0000", 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0679000000000004e-05 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169477 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2387968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.889165483999999, 2024-09-12T07:21:35.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069660978999999998, 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169477 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3299999999999998e-07 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524372+0000", 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1519999999999999e-06 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524379+0000", 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.562e-06 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524383+0000", 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5430000000000004e-06 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593852+0000", 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069468641999999997 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593926+0000", 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4070000000000001e-05 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594031+0000", 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104677 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.488 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169478 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2428928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.889145068000001, 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069830325999999998, 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169478 2024-09-12T07:21:35.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524393+0000", 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.092e-06 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524399+0000", 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3539999999999996e-06 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524404+0000", 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9590000000000003e-06 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594062+0000", 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069658017000000003 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594131+0000", 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9226000000000004e-05 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594220+0000", 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9295999999999993e-05 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169479 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2469888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.889124593999998, 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069996201999999993, 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.493 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169479 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530070+0000", 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056590099999999999 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530075+0000", 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8200000000000002e-06 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530085+0000", 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530093+0000", 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5519999999999999e-06 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594250+0000", 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064157642000000001 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594318+0000", 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.737e-05 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594407+0000", 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9117999999999996e-05 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169502 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 3411968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:35.497 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.882669668999998, 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068036163999999996, 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169502 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:35.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535987+0000", 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051212549999999999 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535993+0000", 2024-09-12T07:21:35.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0639999999999999e-06 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536003+0000", 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0549e-05 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536012+0000", 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2309999999999996e-06 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598753+0000", 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062741665000000002 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598822+0000", 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8689999999999995e-05 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598902+0000", 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9709999999999994e-05 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.501 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:35.502 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:35.502 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:35.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:35.504 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:35.504+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:35.605 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:35.605+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:35.622 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:35.622 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:35.622 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:35.622 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:35.631 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:35.705 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:35.706+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:35.739 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:35.739 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:35.739 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:35.748 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:35.832 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:35.834+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:35.897 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 98.284893045000004, 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:21:35.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:21:35.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:21:35.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:21:35.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.901 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:21:35.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:21:35.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.901 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.901 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:21:35.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:21:35.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.901 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.901 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:21:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:21:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:21:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:21:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:21:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:21:35.902 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 93.149432789000002, 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:21:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:21:35.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:21:35.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 93.145205668000003, 2024-09-12T07:21:35.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:21:35.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:21:35.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 92.417455012999994, 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.912 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:21:35.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:21:35.915 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 92.328478423999996, 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:21:35.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:21:35.918 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:21:35.919 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 92.296927093999997, 2024-09-12T07:21:35.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:21:35.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.923 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.924 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.925 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:21:35.925 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:35.925 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 92.293726558000003, 2024-09-12T07:21:35.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:21:35.925 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.925 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.925 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.925 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:21:35.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 92.290480810000005, 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:21:35.929 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:21:35.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 92.288238978999999, 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:21:35.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:21:35.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.937 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 92.197131775000003, 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:21:35.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.941 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:21:35.942 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 92.193303686999997, 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:35.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.944 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:21:35.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:21:35.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:21:35.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:21:35.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:21:35.945 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:21:35.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.946 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.946 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:21:35.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:21:35.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.946 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.946 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:21:35.946 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:21:35.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:21:35.947 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 69.211390906000005, 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.948 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:35.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:35.949 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.951 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.952 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 69.199577262000005, 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:35.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:35.954 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:21:35.955 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:21:35.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.955 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.955 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.955 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:21:35.955 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:21:35.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.955 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.955 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.955 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.957 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 69.198241214999996, 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:35.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:35.959 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:21:35.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.961 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.961 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:21:35.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:21:35.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.961 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.961 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:21:35.961 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:21:35.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:21:35.962 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165942 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 27.697407201000001, 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93685439599999998, 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:35.963 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165942 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200798+0000", 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.455e-06 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200802+0000", 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2390000000000004e-06 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200814+0000", 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1816e-05 2024-09-12T07:21:35.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200820+0000", 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8760000000000002e-06 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200899+0000", 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8695999999999993e-05 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:35.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200912+0000", 2024-09-12T07:21:35.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3545e-05 2024-09-12T07:21:35.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137326+0000", 2024-09-12T07:21:35.968 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93641327100000005 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137519+0000", 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000193298 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137651+0000", 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013219999999999999 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.970 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165957 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:35.970 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:35.970 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 27.696795324, 2024-09-12T07:21:35.970 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93691399799999997, 2024-09-12T07:21:35.970 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.970 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.970 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:35.970 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:35.970 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:35.970 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165957 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201409+0000", 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1999999999999999e-07 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201412+0000", 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2590000000000001e-06 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201423+0000", 2024-09-12T07:21:35.972 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0547000000000001e-05 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201428+0000", 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.101e-06 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201488+0000", 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9426999999999999e-05 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201498+0000", 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0883999999999999e-05 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138057+0000", 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93655902000000002 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138168+0000", 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011034199999999999 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138322+0000", 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015469799999999999 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165958 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 27.696764252000001, 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.937070242, 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.975 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165958 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.58e-07 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201442+0000", 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.719e-06 2024-09-12T07:21:35.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201447+0000", 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4770000000000002e-06 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201450+0000", 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.219e-06 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201530+0000", 2024-09-12T07:21:35.978 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9833000000000006e-05 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201538+0000", 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.028e-06 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138367+0000", 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93682842899999996 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138429+0000", 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1890000000000005e-05 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138510+0000", 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1289e-05 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165973 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:35.980 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 27.696631626999999, 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93711003900000001, 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165973 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:35.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201573+0000", 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8599999999999996e-07 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201580+0000", 2024-09-12T07:21:35.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3810000000000004e-06 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201590+0000", 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0169999999999999e-05 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201596+0000", 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4110000000000001e-06 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201645+0000", 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9156999999999997e-05 2024-09-12T07:21:35.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201656+0000", 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1112e-05 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138538+0000", 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93688176899999998 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138595+0000", 2024-09-12T07:21:35.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6759000000000003e-05 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138682+0000", 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7293999999999995e-05 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165974 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:35.985 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 27.696593479000001, 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93725339100000005, 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165974 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:35.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201611+0000", 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201614+0000", 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.757e-06 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201620+0000", 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.015e-06 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201623+0000", 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6210000000000001e-06 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201679+0000", 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6017999999999998e-05 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201685+0000", 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2560000000000002e-06 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138699+0000", 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93701424600000005 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138749+0000", 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9951000000000003e-05 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138864+0000", 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115017 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:35.990 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 13.402673518, 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0249796689999999, 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495532+0000", 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6130000000000001e-06 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495542+0000", 2024-09-12T07:21:35.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7990000000000005e-06 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495556+0000", 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4737e-05 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495567+0000", 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0348e-05 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495628+0000", 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0711999999999997e-05 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495638+0000", 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0482999999999999e-05 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520066+0000", 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0244278959999999 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520246+0000", 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000179815 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520510+0000", 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026426599999999998 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:35.995 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:35.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:35.998 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:35.998+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:36.097 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:36.099+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:36.145 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:36.145 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:36.154 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:36.199 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:36.200+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:36.266 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:36.267 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:36.267 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:36.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:36.300 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:36.301+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:36.400 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:36.402+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:36.425 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:36.425 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:36.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:36.425 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:36.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.425 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.944536498000005, 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:21:36.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.944509617999998, 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.430 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:36.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:21:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:21:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:21:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:21:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.434 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.944485598, 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:21:36.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.944457951999993, 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.439 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.442 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:21:36.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:21:36.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:21:36.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.944434490000006, 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.444 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:36.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:21:36.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:21:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:21:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:21:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:21:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.448 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.944411962999993, 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:21:36.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:21:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:21:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:21:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:21:36.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:21:36.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:21:36.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:21:36.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.944390159999998, 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.454 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:36.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.456 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.458 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.937756379999996, 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:21:36.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:21:36.462 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.937736372000003, 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:21:36.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.467 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.931253339999998, 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:36.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:21:36.471 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.931218052999995, 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:21:36.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.931195419999995, 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:21:36.476 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:21:36.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:21:36.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.931141365000002, 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.481 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:36.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:21:36.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.931122423999994, 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.485 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.488 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:36.489 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.931110486999998, 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:36.490 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:21:36.491 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.492 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.493 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.931098872000007, 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.494 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.495 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.924086263999996, 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.498 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.499 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.500 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.502 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.924062548999999, 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.503 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.504 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.506 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.923931691999996, 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.507 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.510 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 92.960260902000002, 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.511 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.516 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:36.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:36.518 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:36.519+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:36.619 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:36.621+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:36.666 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:36.666 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:36.667 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:36.675 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:36.758 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:36.760+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:36.821 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:36.822+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:36.821 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:36.821 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:36.821 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:36.821 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:36.821 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:36.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:36.922 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:36.924+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:36.979 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 104.450447904, 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.980 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.981 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:21:36.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.984 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.598650238000005, 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.985 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:36.986 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.989 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.598526961000005, 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.990 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:21:36.993 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.598323570000005, 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.994 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:36.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:21:36.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:36.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:36.998 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.598187742999997, 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:36.999 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:21:37.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:21:37.002 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.597861086999998, 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:21:37.005 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:21:37.006 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:37.007 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.597822199000007, 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:37.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:21:37.011 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.597455324999999, 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.012 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:21:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.597320468999996, 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:21:37.020 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.596950578000005, 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.021 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:21:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:21:37.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:21:37.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:21:37.025 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:21:37.025 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.025 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.025 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.596760380000006, 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:21:37.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.596540301000005, 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.030 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.032 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:21:37.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.596396635999994, 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:21:37.036 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.596170463999997, 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.039 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.043 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.595970105999996, 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:21:37.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:21:37.046 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.595562223000002, 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.048 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:21:37.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.595096341000001, 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.053 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:21:37.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:21:37.057 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.057 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.057 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.057 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.057 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:37.057 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.593494131, 2024-09-12T07:21:37.057 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:37.058 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.061 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.592960095999999, 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.062 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:21:37.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:21:37.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:21:37.064 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.066 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.336762026000002, 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:37.067 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.069 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:21:37.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:37.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:37.071 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:37.071 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:21:37.071 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:21:37.071 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.071 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.071 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.071 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:37.071 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:37.071 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:37.073 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:37.073 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:37.074+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:37.150 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:21:37.173 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:37.174+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:37.192 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:37.224 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:37.224 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:37.224 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:37.224 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:37.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:37.274 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:37.276+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:37.341 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:37.341 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:37.358 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:37.374 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:37.374 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:37.375 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:37.376+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:37.381 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:37.381 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:37.381 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:37.381 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:37.381 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:37.390 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:37.391 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:37.407 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:37.424 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:37.424 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:37.440 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:37.457 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:37.457 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:37.475 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:37.477+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:37.497 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:37.497 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:37.506 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:37.576 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:37.577+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:37.616 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:37.616 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:37.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:37.677 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:37.679+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:37.773 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:37.777 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:37.779+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:37.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:37.878 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:37.880+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:37.930 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:37.931 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:37.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:37.981 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:37.982+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:38.080 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:38.081+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:38.088 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:38.088 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:38.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:21:38.180 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:38.182+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:38.217 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:38.217 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:38.224 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:38.280 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 6, 7, 3] out_osds: [1, 5, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:21:38.281 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:21:38.281 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-12T07:21:38.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T07:21:38.308 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:38.309+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:38.376 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:38.376 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:38.382 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:38.384+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:38.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:21:38.483 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:38.485+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:38.533 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:38.533 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:38.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:21:38.585 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:38.586+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:21:38.614 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 135, 2024-09-12T07:21:38.614 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "8412ed58-ac5e-4549-b523-1a58cf9f9429", 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "created": "2024-09-12T07:19:10.328057+0000", 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-12T07:21:38.170181+0000", 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "last_up_change": "2024-09-12T07:19:22.177473+0000", 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "last_in_change": "2024-09-12T07:21:21.476884+0000", 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_num": 5799936, 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_set": [ 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "pglog_hardlimit", 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "purged_snapdirs", 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_deletes", 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "sortbitwise" 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_version": 5, 2024-09-12T07:21:38.615 INFO:teuthology.orchestra.run.smithi046.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_snapshot": "", 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_max": 3, 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd": 8, 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "require_min_compat_client": "luminous", 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "min_compat_client": "luminous", 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "require_osd_release": "squid", 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_crimson": false, 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "pools": [ 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 1, 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": ".mgr", 2024-09-12T07:21:38.616 INFO:teuthology.orchestra.run.smithi046.stdout: "create_time": "2024-09-12T07:19:23.507151+0000", 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_names": "hashpspool", 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type": 1, 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 2, 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": 1, 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_rule": 0, 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_count": 0, 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_target": 0, 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "is_stretch_pool": false, 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "object_hash": 2, 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscale_mode": "off", 2024-09-12T07:21:38.617 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": 1, 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num": 1, 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num_target": 1, 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_target": 1, 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_pending": 1, 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "last_pg_merge_meta": { 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "source_pgid": "0.0", 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "ready_epoch": 0, 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_started": 0, 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_clean": 0, 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "source_version": "0'0", 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "target_version": "0'0" 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "last_change": "18", 2024-09-12T07:21:38.618 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend": "0", 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "auid": 0, 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_mode": "selfmanaged", 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_seq": 0, 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_epoch": 0, 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_snaps": [], 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps": "[]", 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_bytes": 0, 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_objects": 0, 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tiers": [], 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tier_of": -1, 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "read_tier": -1, 2024-09-12T07:21:38.619 INFO:teuthology.orchestra.run.smithi046.stdout: "write_tier": -1, 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_mode": "none", 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_bytes": 0, 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_objects": 0, 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_flush_age": 0, 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_evict_age": 0, 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profile": "", 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_params": { 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "none" 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_period": 0, 2024-09-12T07:21:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_count": 0, 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "use_gmt_hitset": true, 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "min_read_recency_for_promote": 0, 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "min_write_recency_for_promote": 0, 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_search_last_n": 0, 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "grade_table": [], 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "stripe_width": 0, 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "expected_num_objects": 0, 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "fast_read": false, 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "options": { 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_max": 32, 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_min": 1 2024-09-12T07:21:38.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "application_metadata": { 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": {} 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "read_balance": { 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "score_type": "Fair distribution", 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "score_acting": 6.9000000953674316, 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "score_stable": 6.9000000953674316, 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "optimal_score": 0.28999999165534973, 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_acting": 2, 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_stable": 2, 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity_weighted": 0.75648117065429688, 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity_weighted": 0.75999999046325684 2024-09-12T07:21:38.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 2, 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": "rbd", 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "create_time": "2024-09-12T07:19:28.593632+0000", 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 8193, 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type": 1, 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 2, 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": 1, 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_rule": 0, 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_count": 0, 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_target": 0, 2024-09-12T07:21:38.623 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "is_stretch_pool": false, 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "object_hash": 2, 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscale_mode": "off", 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": 8, 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num": 8, 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num_target": 8, 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_target": 8, 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_pending": 8, 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "last_pg_merge_meta": { 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "source_pgid": "0.0", 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "ready_epoch": 0, 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_started": 0, 2024-09-12T07:21:38.624 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_clean": 0, 2024-09-12T07:21:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "source_version": "0'0", 2024-09-12T07:21:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "target_version": "0'0" 2024-09-12T07:21:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "last_change": "22", 2024-09-12T07:21:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend": "0", 2024-09-12T07:21:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T07:21:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T07:21:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "auid": 0, 2024-09-12T07:21:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_mode": "selfmanaged", 2024-09-12T07:21:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_seq": 2, 2024-09-12T07:21:38.625 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_epoch": 22, 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_snaps": [], 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps": "[]", 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_bytes": 0, 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_objects": 0, 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tiers": [], 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tier_of": -1, 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "read_tier": -1, 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "write_tier": -1, 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_mode": "none", 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_bytes": 0, 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_objects": 0, 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T07:21:38.626 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_flush_age": 0, 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_evict_age": 0, 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profile": "", 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_params": { 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "none" 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_period": 0, 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_count": 0, 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "use_gmt_hitset": true, 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "min_read_recency_for_promote": 0, 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "min_write_recency_for_promote": 0, 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_search_last_n": 0, 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "grade_table": [], 2024-09-12T07:21:38.627 INFO:teuthology.orchestra.run.smithi046.stdout: "stripe_width": 0, 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "expected_num_objects": 0, 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "fast_read": false, 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "options": {}, 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "application_metadata": { 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd": {} 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "read_balance": { 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "score_type": "Fair distribution", 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "score_acting": 2.6500000953674316, 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "score_stable": 2.6500000953674316, 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "optimal_score": 0.70999997854232788, 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-12T07:21:38.628 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity_weighted": 0.75648117065429688, 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity_weighted": 0.75999999046325684 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 3, 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": "pool_client.0", 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "create_time": "2024-09-12T07:19:40.555421+0000", 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 8193, 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type": 1, 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 2, 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": 1, 2024-09-12T07:21:38.629 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_rule": 0, 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_count": 0, 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_target": 0, 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "is_stretch_pool": false, 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "object_hash": 2, 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscale_mode": "off", 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": 32, 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num": 32, 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_placement_num_target": 32, 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_target": 32, 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num_pending": 32, 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "last_pg_merge_meta": { 2024-09-12T07:21:38.630 INFO:teuthology.orchestra.run.smithi046.stdout: "source_pgid": "0.0", 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "ready_epoch": 0, 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_started": 0, 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "last_epoch_clean": 0, 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "source_version": "0'0", 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "target_version": "0'0" 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "last_change": "135", 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend": "0", 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "auid": 0, 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_mode": "selfmanaged", 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_seq": 47, 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_epoch": 135, 2024-09-12T07:21:38.631 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_snaps": [], 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps": "[]", 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_bytes": 0, 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "quota_max_objects": 0, 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tiers": [], 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tier_of": -1, 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "read_tier": -1, 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "write_tier": -1, 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_mode": "none", 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_bytes": 0, 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "target_max_objects": 0, 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T07:21:38.632 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_flush_age": 0, 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "cache_min_evict_age": 0, 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profile": "", 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_params": { 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "none" 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_period": 0, 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_count": 0, 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "use_gmt_hitset": true, 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "min_read_recency_for_promote": 0, 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "min_write_recency_for_promote": 0, 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "hit_set_search_last_n": 0, 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "grade_table": [], 2024-09-12T07:21:38.633 INFO:teuthology.orchestra.run.smithi046.stdout: "stripe_width": 0, 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "expected_num_objects": 0, 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "fast_read": false, 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "options": {}, 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "application_metadata": { 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd": {} 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "read_balance": { 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "score_type": "Fair distribution", 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "score_acting": 2.4200000762939453, 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "score_stable": 2.4200000762939453, 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "optimal_score": 0.70999997854232788, 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_acting": 1.7200000286102295, 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "raw_score_stable": 1.7200000286102295, 2024-09-12T07:21:38.634 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity_weighted": 0.75648117065429688, 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "average_primary_affinity_weighted": 0.75999999046325684 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "osds": [ 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 0, 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "710c25c3-92d3-4b51-a7fc-670879faa69c", 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:21:38.635 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 121, 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6816", 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6817", 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6818", 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.637 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6819", 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6822", 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.638 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6823", 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6820", 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.639 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6821", 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4081287959 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6817/4081287959", 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6819/4081287959", 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6823/4081287959", 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6821/4081287959", 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 1, 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "04479191-c52e-4b60-aa0d-3e911f869062", 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 0, 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 0, 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 99, 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6824", 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6825", 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6826", 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6827", 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:38.644 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6830", 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6831", 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.645 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6828", 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6829", 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3980488213 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.646 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6825/3980488213", 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6827/3980488213", 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3980488213", 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3980488213", 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 2, 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "ba673e51-96aa-424b-a4df-26432d633ebe", 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:38.647 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 121, 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.648 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6800", 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6801", 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6802", 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6803", 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6806", 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6807", 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6804", 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6805", 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 656802655 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6801/656802655", 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6803/656802655", 2024-09-12T07:21:38.652 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6807/656802655", 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6805/656802655", 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 3, 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "c516e59c-834c-4fb0-9ff8-bcc80b21d28a", 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:21:38.653 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 122, 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6808", 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6809", 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6810", 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6811", 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6814", 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6815", 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6812", 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.657 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6813", 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3848260327 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.46:6809/3848260327", 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.46:6811/3848260327", 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.46:6815/3848260327", 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.46:6813/3848260327", 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 4, 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "450a70cd-ebb2-48ed-a766-184f58e7032e", 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 0, 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 0, 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 0.051849365234375, 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 121, 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:38.659 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6816", 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6817", 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6818", 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6819", 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.661 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6822", 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6823", 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.662 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6820", 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6821", 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3464256744 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6817/3464256744", 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6819/3464256744", 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6823/3464256744", 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6821/3464256744", 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 5, 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "a4efa333-bf57-45e7-a281-1d3bc9d2e82b", 2024-09-12T07:21:38.664 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 0, 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 0, 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 97, 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6824", 2024-09-12T07:21:38.665 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6825", 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6826", 2024-09-12T07:21:38.666 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6827", 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6830", 2024-09-12T07:21:38.667 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6831", 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.668 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6828", 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6829", 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1789414693 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6825/1789414693", 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6827/1789414693", 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6831/1789414693", 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6829/1789414693", 2024-09-12T07:21:38.669 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 6, 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "bcaad352-f9ee-426c-bcbf-39c8d031aae0", 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 0, 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:38.670 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 100, 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6800", 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.671 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6801", 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6802", 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6804", 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6810", 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6813", 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6806", 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6808", 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2576774646 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6801/2576774646", 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6804/2576774646", 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6813/2576774646", 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6808/2576774646", 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:38.675 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 7, 2024-09-12T07:21:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "uuid": "97e5ef36-58a4-4667-9f9e-b8ced5904d69", 2024-09-12T07:21:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-12T07:21:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-12T07:21:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "weight": 1, 2024-09-12T07:21:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_affinity": 1, 2024-09-12T07:21:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_begin": 0, 2024-09-12T07:21:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "last_clean_end": 0, 2024-09-12T07:21:38.676 INFO:teuthology.orchestra.run.smithi046.stdout: "up_from": 14, 2024-09-12T07:21:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "up_thru": 122, 2024-09-12T07:21:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "down_at": 0, 2024-09-12T07:21:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "lost_at": 0, 2024-09-12T07:21:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addrs": { 2024-09-12T07:21:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6803", 2024-09-12T07:21:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6805", 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addrs": { 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6807", 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6809", 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addrs": { 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6814", 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6815", 2024-09-12T07:21:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addrs": { 2024-09-12T07:21:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T07:21:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T07:21:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6811", 2024-09-12T07:21:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.64:6812", 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 368918691 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "public_addr": "172.21.15.64:6805/368918691", 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_addr": "172.21.15.64:6809/368918691", 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_back_addr": "172.21.15.64:6815/368918691", 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "heartbeat_front_addr": "172.21.15.64:6812/368918691", 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "state": [ 2024-09-12T07:21:38.681 INFO:teuthology.orchestra.run.smithi046.stdout: "exists", 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "up" 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_xinfo": [ 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 0, 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:21:38.682 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:18.862785+0000", 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 1, 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 65536, 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.059718+0000", 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 2, 2024-09-12T07:21:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:21:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:18.575165+0000", 2024-09-12T07:21:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:38.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 3, 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:18.847666+0000", 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 4, 2024-09-12T07:21:38.685 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 65536, 2024-09-12T07:21:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.471904+0000", 2024-09-12T07:21:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 5, 2024-09-12T07:21:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:38.686 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 65536, 2024-09-12T07:21:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.061712+0000", 2024-09-12T07:21:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 6, 2024-09-12T07:21:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:38.687 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:21:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.541383+0000", 2024-09-12T07:21:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": 7, 2024-09-12T07:21:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "down_stamp": "0.000000", 2024-09-12T07:21:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_probability": 0, 2024-09-12T07:21:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "laggy_interval": 0, 2024-09-12T07:21:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "features": 4540719139924082687, 2024-09-12T07:21:38.688 INFO:teuthology.orchestra.run.smithi046.stdout: "old_weight": 0, 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: "last_purged_snaps_scrub": "2024-09-12T07:19:19.072272+0000", 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: "dead_epoch": 0 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_upmap": [ 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: "pgid": "3.a", 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: "osds": [ 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: 4, 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: 1 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.689 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_upmap_items": [], 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_upmap_primaries": [], 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_temp": [], 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "primary_temp": [], 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklist": {}, 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "range_blocklist": {}, 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "erasure_code_profiles": { 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "default": { 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-failure-domain": "osd", 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "k": "2", 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "m": "1", 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "plugin": "jerasure", 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: "technique": "reed_sol_van" 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: "removed_snaps_queue": [ 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 3, 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: "snaps": [ 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: "begin": 26, 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: "length": 6 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: "new_removed_snaps": [ 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.691 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 3, 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: "snaps": [ 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: "begin": 26, 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: "length": 1 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: "new_purged_snaps": [], 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_node_flags": {}, 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: "device_class_flags": {}, 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_mode": { 2024-09-12T07:21:38.692 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_mode_enabled": false, 2024-09-12T07:21:38.693 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_bucket_count": 0, 2024-09-12T07:21:38.693 INFO:teuthology.orchestra.run.smithi046.stdout: "degraded_stretch_mode": 0, 2024-09-12T07:21:38.693 INFO:teuthology.orchestra.run.smithi046.stdout: "recovering_stretch_mode": 0, 2024-09-12T07:21:38.693 INFO:teuthology.orchestra.run.smithi046.stdout: "stretch_mode_bucket": 0 2024-09-12T07:21:38.693 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:38.693 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:38.695 INFO:tasks.thrashosds.thrasher:j is {'epoch': 135, 'fsid': '8412ed58-ac5e-4549-b523-1a58cf9f9429', 'created': '2024-09-12T07:19:10.328057+0000', 'modified': '2024-09-12T07:21:38.170181+0000', 'last_up_change': '2024-09-12T07:19:22.177473+0000', 'last_in_change': '2024-09-12T07:21:21.476884+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T07:19:23.507151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7564811706542969, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.7599999904632568}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T07:19:28.593632+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.7099999785423279, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.7564811706542969, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.7599999904632568}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T07:19:40.555421+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 135, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.4200000762939453, 'score_stable': 2.4200000762939453, 'optimal_score': 0.7099999785423279, 'raw_score_acting': 1.7200000286102295, 'raw_score_stable': 1.7200000286102295, 'primary_affinity_weighted': 0.7564811706542969, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.7599999904632568}}], 'osds': [{'osd': 0, 'uuid': '710c25c3-92d3-4b51-a7fc-670879faa69c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6817', 'nonce': 4081287959}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6818', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 4081287959}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6822', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6823', 'nonce': 4081287959}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 4081287959}, {'type': 'v1', 'addr': '172.21.15.46:6821', 'nonce': 4081287959}]}, 'public_addr': '172.21.15.46:6817/4081287959', 'cluster_addr': '172.21.15.46:6819/4081287959', 'heartbeat_back_addr': '172.21.15.46:6823/4081287959', 'heartbeat_front_addr': '172.21.15.46:6821/4081287959', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '04479191-c52e-4b60-aa0d-3e911f869062', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6824', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 3980488213}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6827', 'nonce': 3980488213}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3980488213}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6828', 'nonce': 3980488213}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3980488213}]}, 'public_addr': '172.21.15.46:6825/3980488213', 'cluster_addr': '172.21.15.46:6827/3980488213', 'heartbeat_back_addr': '172.21.15.46:6831/3980488213', 'heartbeat_front_addr': '172.21.15.46:6829/3980488213', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'ba673e51-96aa-424b-a4df-26432d633ebe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6801', 'nonce': 656802655}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6802', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 656802655}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6806', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 656802655}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 656802655}, {'type': 'v1', 'addr': '172.21.15.46:6805', 'nonce': 656802655}]}, 'public_addr': '172.21.15.46:6801/656802655', 'cluster_addr': '172.21.15.46:6803/656802655', 'heartbeat_back_addr': '172.21.15.46:6807/656802655', 'heartbeat_front_addr': '172.21.15.46:6805/656802655', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c516e59c-834c-4fb0-9ff8-bcc80b21d28a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 122, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6809', 'nonce': 3848260327}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6810', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 3848260327}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6814', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 3848260327}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 3848260327}, {'type': 'v1', 'addr': '172.21.15.46:6813', 'nonce': 3848260327}]}, 'public_addr': '172.21.15.46:6809/3848260327', 'cluster_addr': '172.21.15.46:6811/3848260327', 'heartbeat_back_addr': '172.21.15.46:6815/3848260327', 'heartbeat_front_addr': '172.21.15.46:6813/3848260327', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '450a70cd-ebb2-48ed-a766-184f58e7032e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.051849365234375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6816', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6817', 'nonce': 3464256744}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6818', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6819', 'nonce': 3464256744}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6822', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6823', 'nonce': 3464256744}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6820', 'nonce': 3464256744}, {'type': 'v1', 'addr': '172.21.15.64:6821', 'nonce': 3464256744}]}, 'public_addr': '172.21.15.64:6817/3464256744', 'cluster_addr': '172.21.15.64:6819/3464256744', 'heartbeat_back_addr': '172.21.15.64:6823/3464256744', 'heartbeat_front_addr': '172.21.15.64:6821/3464256744', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a4efa333-bf57-45e7-a281-1d3bc9d2e82b', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6824', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6825', 'nonce': 1789414693}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6826', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6827', 'nonce': 1789414693}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6830', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6831', 'nonce': 1789414693}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6828', 'nonce': 1789414693}, {'type': 'v1', 'addr': '172.21.15.64:6829', 'nonce': 1789414693}]}, 'public_addr': '172.21.15.64:6825/1789414693', 'cluster_addr': '172.21.15.64:6827/1789414693', 'heartbeat_back_addr': '172.21.15.64:6831/1789414693', 'heartbeat_front_addr': '172.21.15.64:6829/1789414693', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'bcaad352-f9ee-426c-bcbf-39c8d031aae0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6800', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6801', 'nonce': 2576774646}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6802', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6804', 'nonce': 2576774646}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6810', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6813', 'nonce': 2576774646}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6806', 'nonce': 2576774646}, {'type': 'v1', 'addr': '172.21.15.64:6808', 'nonce': 2576774646}]}, 'public_addr': '172.21.15.64:6801/2576774646', 'cluster_addr': '172.21.15.64:6804/2576774646', 'heartbeat_back_addr': '172.21.15.64:6813/2576774646', 'heartbeat_front_addr': '172.21.15.64:6808/2576774646', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '97e5ef36-58a4-4667-9f9e-b8ced5904d69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 122, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6803', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6805', 'nonce': 368918691}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6807', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6809', 'nonce': 368918691}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6814', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6815', 'nonce': 368918691}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6811', 'nonce': 368918691}, {'type': 'v1', 'addr': '172.21.15.64:6812', 'nonce': 368918691}]}, 'public_addr': '172.21.15.64:6805/368918691', 'cluster_addr': '172.21.15.64:6809/368918691', 'heartbeat_back_addr': '172.21.15.64:6815/368918691', 'heartbeat_front_addr': '172.21.15.64:6812/368918691', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:18.862785+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.059718+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:18.575165+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:18.847666+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.471904+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.061712+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.541383+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T07:19:19.072272+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.a', 'osds': [4, 1]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 26, 'length': 6}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 26, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T07:21:38.695 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-12T07:21:38.696 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:38.696 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:38.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:21:38.697 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:38.695+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:38.795 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:38.797+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:38.842 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:38.842 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:38.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:38.896 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:38.898+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:38.958 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:38.958 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:38.966 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:39.007 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:39.009+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:39.098 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:39.100+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:39.113 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:39.113 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:39.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:39.205 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:39.207+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:39.287 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:39.288 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:39.297 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:39.300 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:39.302+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:39.401 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:39.402+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:39.411 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:39.411 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:39.419 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:39.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:39.512 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:39.514+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:39.567 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:39.567 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:39.577 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:39.593 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:39.593 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:39.603 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:39.605+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:39.610 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:39.610 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:39.626 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:39.642 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:39.642 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:39.659 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:39.659 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:39.676 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:39.686 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:39.686 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:39.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:39.686 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:39.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.686 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:21:39.686 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:39.686 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 102.073626369, 2024-09-12T07:21:39.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:21:39.686 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:21:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:21:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:21:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:21:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:21:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:21:39.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:21:39.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:21:39.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:21:39.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:21:39.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.938166112999994, 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:21:39.691 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:21:39.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.695 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.933938991999995, 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.696 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:39.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:21:39.700 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.206188337, 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:21:39.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:21:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:21:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:21:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:21:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.117211748000003, 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.705 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:21:39.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.709 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.085660418000003, 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:39.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.082459881999995, 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:21:39.714 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:21:39.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.718 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.079214133999997, 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:21:39.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:21:39.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.076972303000005, 2024-09-12T07:21:39.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:21:39.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:21:39.727 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 95.985865098999994, 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:21:39.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:21:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 95.982037011000003, 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:21:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 73.000124229999997, 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:21:39.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.741 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 72.988310585999997, 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:21:39.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:21:39.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:21:39.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:21:39.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:21:39.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:21:39.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 72.986974539000002, 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:21:39.746 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:21:39.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165942 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 31.486140525, 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93685439599999998, 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165942 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.751 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200798+0000", 2024-09-12T07:21:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.455e-06 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200802+0000", 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2390000000000004e-06 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200814+0000", 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1816e-05 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200820+0000", 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8760000000000002e-06 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200899+0000", 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8695999999999993e-05 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200912+0000", 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3545e-05 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137326+0000", 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93641327100000005 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137519+0000", 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000193298 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137651+0000", 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013219999999999999 2024-09-12T07:21:39.755 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.756 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.756 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.756 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165957 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:39.756 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:39.756 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 31.485528647999999, 2024-09-12T07:21:39.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93691399799999997, 2024-09-12T07:21:39.756 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.756 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.756 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:39.756 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165957 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:39.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201409+0000", 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1999999999999999e-07 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201412+0000", 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2590000000000001e-06 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201423+0000", 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0547000000000001e-05 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201428+0000", 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.101e-06 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201488+0000", 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9426999999999999e-05 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201498+0000", 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0883999999999999e-05 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138057+0000", 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93655902000000002 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138168+0000", 2024-09-12T07:21:39.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011034199999999999 2024-09-12T07:21:39.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138322+0000", 2024-09-12T07:21:39.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015469799999999999 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165958 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 31.485497576, 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.937070242, 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:39.762 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165958 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:39.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.58e-07 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201442+0000", 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.719e-06 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201447+0000", 2024-09-12T07:21:39.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4770000000000002e-06 2024-09-12T07:21:39.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201450+0000", 2024-09-12T07:21:39.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.219e-06 2024-09-12T07:21:39.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201530+0000", 2024-09-12T07:21:39.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9833000000000006e-05 2024-09-12T07:21:39.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:39.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201538+0000", 2024-09-12T07:21:39.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.028e-06 2024-09-12T07:21:39.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138367+0000", 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93682842899999996 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138429+0000", 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1890000000000005e-05 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138510+0000", 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1289e-05 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.767 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165973 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 31.485364951000001, 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93711003900000001, 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:39.768 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165973 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201573+0000", 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8599999999999996e-07 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201580+0000", 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3810000000000004e-06 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201590+0000", 2024-09-12T07:21:39.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0169999999999999e-05 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201596+0000", 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4110000000000001e-06 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201645+0000", 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9156999999999997e-05 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:39.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201656+0000", 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1112e-05 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138538+0000", 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93688176899999998 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138595+0000", 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6759000000000003e-05 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138682+0000", 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7293999999999995e-05 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165974 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 31.485326803, 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93725339100000005, 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.773 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165974 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201611+0000", 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201614+0000", 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.757e-06 2024-09-12T07:21:39.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201620+0000", 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.015e-06 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201623+0000", 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6210000000000001e-06 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201679+0000", 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6017999999999998e-05 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201685+0000", 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2560000000000002e-06 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138699+0000", 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93701424600000005 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138749+0000", 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9951000000000003e-05 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138864+0000", 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115017 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-12T07:21:39.778 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 17.191406841999999, 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0249796689999999, 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:39.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495532+0000", 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6130000000000001e-06 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495542+0000", 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7990000000000005e-06 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495556+0000", 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4737e-05 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495567+0000", 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0348e-05 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495628+0000", 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0711999999999997e-05 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495638+0000", 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0482999999999999e-05 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520066+0000", 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0244278959999999 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:39.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520246+0000", 2024-09-12T07:21:39.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000179815 2024-09-12T07:21:39.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:39.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:39.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:39.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520510+0000", 2024-09-12T07:21:39.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026426599999999998 2024-09-12T07:21:39.783 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.783 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.783 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.783 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:39.784 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:39.784 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:39.785 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:39.785 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:39.786 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:39.786 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:39.786 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:39.787+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:39.787 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:39.885 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:39.887+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:39.934 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:39.935 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:39.935 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:39.943 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:39.986 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:39.988+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:40.050 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:40.050 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:40.050 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:40.051 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:40.051 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:40.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:40.107 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:40.109+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:40.167 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:40.167 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:40.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.686611986, 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:21:40.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.169 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:40.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:40.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:21:40.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:21:40.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:21:40.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:21:40.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:21:40.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:21:40.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:21:40.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:21:40.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:21:40.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:21:40.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:21:40.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:21:40.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:21:40.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:21:40.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.173 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.173 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.173 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:40.173 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.686585106, 2024-09-12T07:21:40.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:21:40.173 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.174 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.174 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.174 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.174 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:21:40.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.174 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:40.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:21:40.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:21:40.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:21:40.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:21:40.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:21:40.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:21:40.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:21:40.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:21:40.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:21:40.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:21:40.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.686561086, 2024-09-12T07:21:40.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:21:40.179 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.179 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.179 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.179 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.179 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:21:40.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.179 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:40.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:40.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:21:40.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:21:40.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:21:40.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:21:40.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:21:40.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:21:40.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:21:40.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:21:40.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:21:40.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:21:40.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:21:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:21:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:21:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:21:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.183 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.68653344000001, 2024-09-12T07:21:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:21:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.184 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:21:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:21:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:21:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:21:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:21:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:21:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:21:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:21:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.686509978, 2024-09-12T07:21:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:21:40.189 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:21:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:21:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:21:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:21:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:21:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:21:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:21:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:21:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:21:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:21:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:21:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:21:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:21:40.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:21:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.686487451, 2024-09-12T07:21:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:21:40.195 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:21:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:21:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:21:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:21:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:21:40.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:21:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:21:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:21:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:21:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:21:40.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:21:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:21:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:21:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:21:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.200 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.686465648, 2024-09-12T07:21:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:21:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.201 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:21:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:21:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:21:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:21:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:21:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:21:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:21:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.204 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:21:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:21:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:21:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:21:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:21:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:21:40.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.67983186799999, 2024-09-12T07:21:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:21:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:21:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:21:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:21:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:21:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:21:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:21:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:21:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:21:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:21:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:21:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:21:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.67981186, 2024-09-12T07:21:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:21:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:21:40.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:21:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:21:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:21:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:21:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:21:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:21:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:21:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:21:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:21:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:21:40.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:21:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:21:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:21:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:21:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.216 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.673328828, 2024-09-12T07:21:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:21:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.217 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:21:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:21:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:21:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:21:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:21:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:21:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:21:40.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:21:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:21:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:21:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:21:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:21:40.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:21:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:21:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.222 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.67329354100001, 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:21:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:21:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:21:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:21:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:21:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:21:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:21:40.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:21:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:21:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:21:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:21:40.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:21:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:40.228 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.67327090800001, 2024-09-12T07:21:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:21:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:21:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.230 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:21:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:21:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:21:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:21:40.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:21:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:21:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:21:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:21:40.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:21:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:21:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:21:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:21:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:21:40.233 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.673216853, 2024-09-12T07:21:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:21:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.234 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:21:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.236 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:21:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:21:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:21:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:21:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:21:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:21:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:21:40.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:21:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:21:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:21:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:21:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:21:40.239 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.67319791200001, 2024-09-12T07:21:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:21:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.241 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:21:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.245 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.673185975, 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:21:40.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:21:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:21:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:21:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:21:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.67317436, 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.250 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:40.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:21:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:21:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:21:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:21:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.66616175199999, 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:21:40.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:21:40.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.666138037, 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.259 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:40.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.263 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 100.66600717999999, 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:21:40.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:21:40.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.267 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.702336389999999, 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:21:40.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.273 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:40.274 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:21:40.276 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:40.276 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:40.276+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:40.376 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:40.377+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:40.423 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:40.423 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:40.423 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:40.423 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:40.431 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:40.477 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:40.478+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:40.538 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:40.538 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:40.538 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:40.538 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:40.538 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:40.547 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:40.591 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:40.593+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:40.656 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:40.656 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:40.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:40.656 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:40.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.656 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:21:40.656 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:40.656 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 108.127142656, 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:40.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:21:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:21:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:21:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:21:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:21:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:21:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.660 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.660 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:21:40.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:21:40.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.660 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:40.660 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:21:40.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:21:40.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.660 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:21:40.661 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.27534498999999, 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.662 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.663 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:21:40.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.663 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:40.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:21:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:21:40.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:21:40.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:21:40.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:21:40.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:21:40.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:21:40.667 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.667 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.667 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.668 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.668 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:40.668 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.27522171299999, 2024-09-12T07:21:40.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:21:40.668 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.668 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.669 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.669 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.669 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.669 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:21:40.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.669 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:40.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:40.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:21:40.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:21:40.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:21:40.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:21:40.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:21:40.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:21:40.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:21:40.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.674 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.674 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:21:40.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:21:40.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.674 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.674 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.675 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:40.675 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.27501832199999, 2024-09-12T07:21:40.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:21:40.675 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.675 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.675 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.675 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.675 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.675 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:21:40.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.676 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:40.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:40.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:21:40.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:21:40.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:21:40.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:21:40.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:21:40.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:21:40.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:21:40.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:21:40.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:21:40.680 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.680 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.274882495, 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.681 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:21:40.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.274555839, 2024-09-12T07:21:40.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:21:40.686 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.686 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.686 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.686 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.686 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.686 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.686 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:21:40.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.686 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:40.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:21:40.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:21:40.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.274516951, 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.691 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:21:40.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:21:40.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:21:40.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:21:40.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:21:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.274150077, 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.696 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:21:40.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:21:40.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:21:40.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:21:40.700 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.700 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.701 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.701 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.701 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:40.701 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.274015221, 2024-09-12T07:21:40.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:21:40.701 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.701 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:21:40.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:21:40.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:21:40.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:21:40.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:21:40.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:21:40.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.706 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.706 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:21:40.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:21:40.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:21:40.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:21:40.708 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.708 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.708 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.708 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.708 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:40.708 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.27364532999999, 2024-09-12T07:21:40.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:21:40.709 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.709 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.709 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.709 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.709 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.709 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.709 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:21:40.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.709 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.710 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:21:40.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:21:40.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.711 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.711 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.711 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:21:40.711 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:21:40.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.712 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.713 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.273455132, 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.714 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:21:40.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:21:40.717 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.27323505299999, 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.719 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:21:40.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.719 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.719 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:40.719 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.719 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.719 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:40.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:21:40.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:21:40.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.720 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.720 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:21:40.720 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.721 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.722 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.723 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.723 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:40.723 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.273091388, 2024-09-12T07:21:40.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:21:40.723 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.723 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.723 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.723 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:21:40.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:21:40.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:21:40.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:21:40.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:21:40.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:21:40.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.272865216, 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:21:40.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:21:40.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:21:40.733 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.733 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.733 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.733 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.733 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:40.733 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.272664858, 2024-09-12T07:21:40.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:21:40.733 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:21:40.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:21:40.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:21:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:21:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:21:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:21:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:21:40.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.738 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.738 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.738 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.739 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:40.739 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.272256975, 2024-09-12T07:21:40.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:21:40.739 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.739 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.739 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.739 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.739 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.739 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:21:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:21:40.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:21:40.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:21:40.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:21:40.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.743 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.271791093, 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:21:40.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.745 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:40.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:40.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:21:40.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:21:40.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.270188883, 2024-09-12T07:21:40.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:21:40.749 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.749 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.749 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.749 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.749 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:21:40.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.750 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:40.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:21:40.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.754 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.754 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:40.754 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.269654848, 2024-09-12T07:21:40.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:21:40.754 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.754 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.754 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.754 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.754 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.754 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.755 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:21:40.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.755 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:40.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:40.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:21:40.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:21:40.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:21:40.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:21:40.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:21:40.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:21:40.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:21:40.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:21:40.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.759 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.760 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.760 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:40.760 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:40.760 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 101.01345677800001, 2024-09-12T07:21:40.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:21:40.760 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:40.760 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:40.760 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:40.760 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:40.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:40.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:40.764 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:40.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:40.766 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:40.768+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:40.866 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:40.868+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:40.914 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:40.914 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:40.914 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:40.914 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:40.923 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:41.006 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:41.008+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:41.068 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:41.069+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:41.069 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:41.069 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:41.069 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:41.069 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:41.069 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:41.078 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:41.168 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:41.170+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:21:41.234 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:41.235 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:41.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:41.235 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:41.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.235 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.235 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:41.235 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.686293704999997, 2024-09-12T07:21:41.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:21:41.235 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.235 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.235 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.236 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:21:41.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.239 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.686273168, 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.240 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:21:41.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.686253164999997, 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:21:41.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:21:41.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:21:41.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:21:41.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:21:41.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.248 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.248 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:21:41.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:21:41.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.686233216000005, 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:21:41.249 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:21:41.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:21:41.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:21:41.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:21:41.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:21:41.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:21:41.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.686212347999998, 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.254 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:41.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:21:41.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.686190647000004, 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:21:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:21:41.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:21:41.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:21:41.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:21:41.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:21:41.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:21:41.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.686170207000004, 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:21:41.263 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.264 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.264 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.264 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.264 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.264 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.264 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:21:41.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.264 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:41.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:21:41.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.686142485999994, 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.268 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:41.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.272 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.686119551000004, 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:21:41.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.686099251000002, 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.277 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.278 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.278 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.278 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.278 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:21:41.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.278 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:41.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:21:41.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:21:41.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:21:41.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:21:41.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:21:41.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:21:41.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.686078975000001, 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.282 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.686058815999999, 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.286 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:21:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:21:41.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.685921617999995, 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:21:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:21:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:21:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:21:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:21:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:21:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.685898969999997, 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:21:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.679436984000006, 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.679207347000002, 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:21:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:21:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:21:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.672283656000005, 2024-09-12T07:21:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:21:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.671924073, 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:21:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 97.671844086999997, 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:21:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 19.742496836000001, 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0005192119999999, 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:41.323 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492882+0000", 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492888+0000", 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:21:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492904+0000", 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5628e-05 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492912+0000", 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4039999999999995e-06 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492962+0000", 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9425999999999997e-05 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492969+0000", 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1729999999999998e-06 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.493388+0000", 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0004188510000001 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.493399+0000", 2024-09-12T07:21:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1571e-05 2024-09-12T07:21:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:41.328 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:41.330 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:41.331+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:41.330 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:41.430 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:41.432+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:41.440 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:41.441 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:41.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:41.531 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:41.533+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:41.556 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:41.556 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:41.556 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:41.556 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:41.557 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:41.565 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:41.649 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:41.651+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:41.713 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:41.713 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:148945 3.10 3:0cac0a45:::rbd_data.1160161ce0c8.000000000000009d:head [delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.280433564000006, 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89878390500000005, 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.714 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 148945 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433803+0000", 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.164e-06 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433807+0000", 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5000000000000001e-06 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433816+0000", 2024-09-12T07:21:41.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1859999999999998e-06 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433822+0000", 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5300000000000004e-06 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433886+0000", 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4510000000000004e-05 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:41.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433896+0000", 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0012e-05 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329359+0000", 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89546257399999996 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329585+0000", 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022641800000000001 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329712+0000", 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126339 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.330704+0000", 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099219900000000003 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332513+0000", 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001808689 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332544+0000", 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1186999999999998e-05 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332585+0000", 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1597000000000002e-05 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170165 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1101824~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.152872643999999, 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073287456000000001, 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170165 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561363+0000", 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.96e-07 2024-09-12T07:21:41.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561365+0000", 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.181e-06 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561371+0000", 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561377+0000", 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.536e-06 2024-09-12T07:21:41.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634513+0000", 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073136229999999997 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634548+0000", 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5206999999999997e-05 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634650+0000", 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101678 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170166 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.152839303999997, 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073346145000000001, 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.725 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170166 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:41.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561399+0000", 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4049999999999998e-06 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561405+0000", 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5370000000000003e-06 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561410+0000", 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2889999999999997e-06 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634666+0000", 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073255526000000001 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634701+0000", 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5197999999999997e-05 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634742+0000", 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0757999999999998e-05 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170167 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1183744~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.152816813000001, 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073410903, 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170167 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561419+0000", 2024-09-12T07:21:41.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3000000000000002e-07 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561421+0000", 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1059999999999998e-06 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561427+0000", 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0970000000000002e-06 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561432+0000", 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1120000000000004e-06 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634756+0000", 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073324571000000005 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634790+0000", 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3081999999999999e-05 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634829+0000", 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9604999999999998e-05 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170168 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1224704~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.152796563000003, 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073476793999999998, 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170168 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561439+0000", 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561441+0000", 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8920000000000001e-06 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561447+0000", 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2360000000000001e-06 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561452+0000", 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9810000000000003e-06 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634843+0000", 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073391434000000005 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634876+0000", 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2869e-05 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634915+0000", 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9001999999999997e-05 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170169 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1265664~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.152776684999999, 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073551819000000004, 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170169 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561459+0000", 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.77e-07 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561460+0000", 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.733e-06 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561468+0000", 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561474+0000", 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4829999999999999e-06 2024-09-12T07:21:41.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634930+0000", 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073456244000000004 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634969+0000", 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9687000000000001e-05 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635010+0000", 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0685000000000001e-05 2024-09-12T07:21:41.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170170 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1306624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.152754289999997, 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073616897000000001, 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170170 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2e-07 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561483+0000", 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0619999999999999e-06 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561491+0000", 2024-09-12T07:21:41.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7880000000000007e-06 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561497+0000", 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5480000000000001e-06 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635025+0000", 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073528643000000005 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635058+0000", 2024-09-12T07:21:41.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3077999999999997e-05 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635098+0000", 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170171 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1347584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.152731525, 2024-09-12T07:21:41.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073682407000000005, 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170171 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561506+0000", 2024-09-12T07:21:41.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.441e-06 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561535+0000", 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8827999999999999e-05 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561542+0000", 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7100000000000001e-06 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635112+0000", 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073569967999999999 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635146+0000", 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4413999999999999e-05 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635186+0000", 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9625000000000001e-05 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170172 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.152684170999997, 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073720519999999998, 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170172 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.752 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1500000000000005e-07 2024-09-12T07:21:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561554+0000", 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561563+0000", 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7760000000000003e-06 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561568+0000", 2024-09-12T07:21:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.305e-06 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635200+0000", 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073632141999999998 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635233+0000", 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.239e-05 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635271+0000", 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8688e-05 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170173 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1429504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.152659583000002, 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07378113, 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170173 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561576+0000", 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999998e-07 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561578+0000", 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0389999999999999e-06 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561587+0000", 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0999999999999993e-06 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561594+0000", 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.844e-06 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635286+0000", 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073691799000000002 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635318+0000", 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2240999999999998e-05 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635357+0000", 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8726000000000002e-05 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170174 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1470464~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.152631392000004, 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073841247999999998, 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170174 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561607+0000", 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561613+0000", 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3339999999999998e-06 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561618+0000", 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.203e-06 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635371+0000", 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073752811000000001 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635404+0000", 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3250000000000002e-05 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635445+0000", 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0741000000000002e-05 2024-09-12T07:21:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170175 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1511424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.152609771000002, 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073939900000000003, 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170175 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561626+0000", 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561632+0000", 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8290000000000002e-06 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561642+0000", 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1209999999999996e-06 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561647+0000", 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5119999999999999e-06 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635459+0000", 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073811962999999994 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635521+0000", 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2323000000000006e-05 2024-09-12T07:21:41.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635565+0000", 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3763000000000001e-05 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170176 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1552384~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.152572720999999, 2024-09-12T07:21:41.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.074029015000000004, 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170176 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561663+0000", 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561665+0000", 2024-09-12T07:21:41.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561672+0000", 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4930000000000003e-06 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561677+0000", 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3159999999999996e-06 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635587+0000", 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073909937999999994 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635629+0000", 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2555999999999997e-05 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635691+0000", 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1946000000000006e-05 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170183 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1757184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.150618821999998, 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073321546000000001, 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170183 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563617+0000", 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999999e-07 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563619+0000", 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.153e-06 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563625+0000", 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3160000000000002e-06 2024-09-12T07:21:41.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563632+0000", 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9820000000000002e-06 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636752+0000", 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073120428000000001 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636838+0000", 2024-09-12T07:21:41.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5177000000000001e-05 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636938+0000", 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010008 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170185 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1798144~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:41.778 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.150301323999997, 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073255606000000001, 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170185 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5600000000000005e-07 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563989+0000", 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4993e-05 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563998+0000", 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8710000000000003e-06 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564005+0000", 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8449999999999997e-06 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636991+0000", 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072986350000000005 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637076+0000", 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4248999999999999e-05 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637189+0000", 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011364199999999999 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170186 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1839104~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.150217765999997, 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073312293000000001, 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170186 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564018+0000", 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.99e-07 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564021+0000", 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7410000000000001e-06 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564027+0000", 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5119999999999997e-06 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564033+0000", 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4970000000000001e-06 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637230+0000", 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073197613999999994 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637271+0000", 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1001000000000003e-05 2024-09-12T07:21:41.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637330+0000", 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8328999999999997e-05 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170189 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1921024~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.149882636000001, 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073393609999999998, 2024-09-12T07:21:41.787 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170189 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564353+0000", 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9500000000000003e-07 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564355+0000", 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1409999999999999e-06 2024-09-12T07:21:41.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564362+0000", 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6279999999999999e-06 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564367+0000", 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5520000000000004e-06 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637563+0000", 2024-09-12T07:21:41.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073196285 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637641+0000", 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7051000000000005e-05 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637746+0000", 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010545800000000001 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170191 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1961984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.149599674000001, 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073246585000000003, 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170191 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.792 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564636+0000", 2024-09-12T07:21:41.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6400000000000002e-07 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564639+0000", 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.399e-06 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564646+0000", 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0999999999999998e-06 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564653+0000", 2024-09-12T07:21:41.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9659999999999999e-06 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637785+0000", 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073131357999999994 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637826+0000", 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1742000000000001e-05 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637882+0000", 2024-09-12T07:21:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5455999999999999e-05 2024-09-12T07:21:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170192 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 2002944~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.149569659999997, 2024-09-12T07:21:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073448428999999996, 2024-09-12T07:21:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170192 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564666+0000", 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564668+0000", 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564675+0000", 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2230000000000004e-06 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564680+0000", 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3210000000000004e-06 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637923+0000", 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073243483999999998 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638011+0000", 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7149000000000002e-05 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638114+0000", 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103278 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170198 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 2084864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.149006790999998, 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073259037999999999, 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170198 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565229+0000", 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0200000000000002e-07 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565231+0000", 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2330000000000001e-06 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565238+0000", 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.596e-06 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565243+0000", 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.31e-06 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638324+0000", 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073080720000000002 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638391+0000", 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7037000000000002e-05 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638487+0000", 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6639999999999996e-05 2024-09-12T07:21:41.804 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:41.805 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:41.805 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:41.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:41.806 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:41.807+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:41.807 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:41.907 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:41.909+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:41.959 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:41.959 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:41.959 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:41.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:41.974 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:41.974 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:41.991 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:42.007 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:42.009+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:42.008 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2: {} 2024-09-12T07:21:42.008 INFO:teuthology.orchestra.run.smithi046.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T07:21:42.024 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:42.025 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:42.041 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:42.042 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:42.059 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:42.076 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:42.076 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:42.082 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:42.082 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:42.091 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:42.093 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:42.093 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:42.145 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:42.147+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:42.202 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:42.202 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:42.209 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:42.210+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:42.215 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:21:42.313 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:42.311+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:42.370 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:42.370 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:42.378 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T07:21:42.410 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:42.412+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:42.511 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:42.512+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:42.526 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:42.526 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:42.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T07:21:42.613 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:42.614+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:42.682 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:42.682 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:42.690 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:42.734 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:42.736+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:42.799 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:42.799 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:42.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:42.814 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:42.816+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:42.915 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:42.917+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:42.954 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:42.955 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:42.963 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:43.048 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:21:43.049+0000 7fd5b8d33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33620) UID: 0 2024-09-12T07:21:43.111 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:43.111 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:43.117 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:43.119+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:43.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:43.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T07:21:43.244 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:43.244 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:43.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:43.255 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:43.257+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:43.320 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:43.321+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:43.363 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:43.364 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:43.372 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:43.420 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:43.422+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:43.479 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:43.479 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:43.489 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:43.530 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:43.532+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:43.622 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:43.624+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:43.637 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:43.637 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:43.645 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:43.696 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 6, 7, 3] out_osds: [1, 5, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:21:43.696 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:21:43.696 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T07:21:43.697 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-12T07:21:43.697 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:21:43.723 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:43.725+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:43.726 INFO:tasks.ceph.osd.2:Stopped 2024-09-12T07:21:43.727 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.2 2024-09-12T07:21:43.727 INFO:tasks.thrashosds.thrasher:remote for osd 2 is ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T07:21:43.727 DEBUG:teuthology.orchestra.run.smithi046:> type ceph-objectstore-tool 2024-09-12T07:21:43.753 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:43.753 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:43.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:43.782 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-12T07:21:43.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs 2024-09-12T07:21:43.824 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:43.825+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:43.909 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:43.909 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:43.917 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:43.924 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:43.926+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:44.025 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:44.025 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:44.025 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:44.027+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:44.034 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:44.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:44.125 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:44.127+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:44.149 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:44.206 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is set 2024-09-12T07:21:44.226 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:44.228+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:44.257 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:44.274 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:44.275 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:44.275 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-12T07:21:44.293 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:44.300 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:44.301 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:44.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:44.310 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:44.311 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:44.327 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:44.329+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:44.328 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:44.331 INFO:teuthology.orchestra.run.smithi046.stdout:3.9 2024-09-12T07:21:44.331 INFO:teuthology.orchestra.run.smithi046.stdout:3.4 2024-09-12T07:21:44.331 INFO:teuthology.orchestra.run.smithi046.stdout:3.1f 2024-09-12T07:21:44.331 INFO:teuthology.orchestra.run.smithi046.stdout:3.1a 2024-09-12T07:21:44.331 INFO:teuthology.orchestra.run.smithi046.stdout:3.13 2024-09-12T07:21:44.332 INFO:teuthology.orchestra.run.smithi046.stdout:3.5 2024-09-12T07:21:44.332 INFO:teuthology.orchestra.run.smithi046.stdout:2.3 2024-09-12T07:21:44.332 INFO:teuthology.orchestra.run.smithi046.stdout:3.0 2024-09-12T07:21:44.332 INFO:teuthology.orchestra.run.smithi046.stdout:2.1 2024-09-12T07:21:44.345 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:44.346 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:44.362 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:44.366 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T07:21:44.367 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T07:21:44.421 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:44.421 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:44.421 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:44.421 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:44.428 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:44.429+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:44.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:44.529 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:44.531+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:44.577 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:44.577 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:44.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:44.605 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op export --pgid 3.5 --file /var/log/ceph/exp.3.5.2 2024-09-12T07:21:44.648 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:44.649+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:44.731 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:44.733+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:44.755 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:148945 3.10 3:0cac0a45:::rbd_data.1160161ce0c8.000000000000009d:head [delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-12T07:21:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.322211073999995, 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89878390500000005, 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 148945 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433803+0000", 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.164e-06 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433807+0000", 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5000000000000001e-06 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433816+0000", 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1859999999999998e-06 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433822+0000", 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5300000000000004e-06 2024-09-12T07:21:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433886+0000", 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4510000000000004e-05 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433896+0000", 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0012e-05 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329359+0000", 2024-09-12T07:21:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89546257399999996 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329585+0000", 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022641800000000001 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329712+0000", 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126339 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.330704+0000", 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099219900000000003 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332513+0000", 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001808689 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332544+0000", 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1186999999999998e-05 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332585+0000", 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1597000000000002e-05 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170165 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1101824~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.194650154000001, 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073287456000000001, 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170165 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561363+0000", 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.96e-07 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561365+0000", 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.181e-06 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561371+0000", 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561377+0000", 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.536e-06 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634513+0000", 2024-09-12T07:21:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073136229999999997 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634548+0000", 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5206999999999997e-05 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634650+0000", 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101678 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170166 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.194616814, 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073346145000000001, 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170166 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561399+0000", 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4049999999999998e-06 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561405+0000", 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5370000000000003e-06 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561410+0000", 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2889999999999997e-06 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634666+0000", 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073255526000000001 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634701+0000", 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5197999999999997e-05 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634742+0000", 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0757999999999998e-05 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170167 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1183744~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.194594322999997, 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073410903, 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170167 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561419+0000", 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3000000000000002e-07 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561421+0000", 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1059999999999998e-06 2024-09-12T07:21:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561427+0000", 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0970000000000002e-06 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561432+0000", 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1120000000000004e-06 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634756+0000", 2024-09-12T07:21:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073324571000000005 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634790+0000", 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3081999999999999e-05 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634829+0000", 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9604999999999998e-05 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170168 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1224704~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.194574072999998, 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073476793999999998, 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170168 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561439+0000", 2024-09-12T07:21:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561441+0000", 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8920000000000001e-06 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561447+0000", 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2360000000000001e-06 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561452+0000", 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9810000000000003e-06 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634843+0000", 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073391434000000005 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634876+0000", 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2869e-05 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634915+0000", 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9001999999999997e-05 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170169 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1265664~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.194554195000002, 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073551819000000004, 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170169 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561459+0000", 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.77e-07 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561460+0000", 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.733e-06 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561468+0000", 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561474+0000", 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4829999999999999e-06 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634930+0000", 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073456244000000004 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634969+0000", 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9687000000000001e-05 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635010+0000", 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0685000000000001e-05 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170170 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1306624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.1945318, 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073616897000000001, 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170170 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2e-07 2024-09-12T07:21:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561483+0000", 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0619999999999999e-06 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561491+0000", 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7880000000000007e-06 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561497+0000", 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5480000000000001e-06 2024-09-12T07:21:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635025+0000", 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073528643000000005 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635058+0000", 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3077999999999997e-05 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635098+0000", 2024-09-12T07:21:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170171 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1347584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.194509035000003, 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073682407000000005, 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170171 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561506+0000", 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.441e-06 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561535+0000", 2024-09-12T07:21:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8827999999999999e-05 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561542+0000", 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7100000000000001e-06 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635112+0000", 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073569967999999999 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635146+0000", 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4413999999999999e-05 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635186+0000", 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9625000000000001e-05 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170172 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.194461681, 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073720519999999998, 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170172 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1500000000000005e-07 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561554+0000", 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561563+0000", 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7760000000000003e-06 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561568+0000", 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.305e-06 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635200+0000", 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073632141999999998 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635233+0000", 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.239e-05 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635271+0000", 2024-09-12T07:21:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8688e-05 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170173 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1429504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.194437092999998, 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07378113, 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170173 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561576+0000", 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999998e-07 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561578+0000", 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0389999999999999e-06 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561587+0000", 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0999999999999993e-06 2024-09-12T07:21:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561594+0000", 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.844e-06 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635286+0000", 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073691799000000002 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635318+0000", 2024-09-12T07:21:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2240999999999998e-05 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635357+0000", 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8726000000000002e-05 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170174 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1470464~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.194408901999999, 2024-09-12T07:21:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073841247999999998, 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170174 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561607+0000", 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561613+0000", 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3339999999999998e-06 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561618+0000", 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.203e-06 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635371+0000", 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073752811000000001 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635404+0000", 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3250000000000002e-05 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635445+0000", 2024-09-12T07:21:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0741000000000002e-05 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170175 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1511424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.194387280999997, 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073939900000000003, 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170175 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561626+0000", 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561632+0000", 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8290000000000002e-06 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561642+0000", 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1209999999999996e-06 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561647+0000", 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5119999999999999e-06 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635459+0000", 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073811962999999994 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635521+0000", 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2323000000000006e-05 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635565+0000", 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3763000000000001e-05 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170176 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1552384~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.194350231000001, 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.074029015000000004, 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170176 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561663+0000", 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561665+0000", 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561672+0000", 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4930000000000003e-06 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561677+0000", 2024-09-12T07:21:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3159999999999996e-06 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635587+0000", 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073909937999999994 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635629+0000", 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2555999999999997e-05 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635691+0000", 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1946000000000006e-05 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170183 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1757184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.192396332000001, 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073321546000000001, 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170183 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563617+0000", 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999999e-07 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563619+0000", 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.153e-06 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563625+0000", 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3160000000000002e-06 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563632+0000", 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9820000000000002e-06 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636752+0000", 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073120428000000001 2024-09-12T07:21:44.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636838+0000", 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5177000000000001e-05 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636938+0000", 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010008 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170185 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1798144~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.192078834, 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073255606000000001, 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170185 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.821 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:44.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5600000000000005e-07 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563989+0000", 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4993e-05 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563998+0000", 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8710000000000003e-06 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564005+0000", 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8449999999999997e-06 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636991+0000", 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072986350000000005 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637076+0000", 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4248999999999999e-05 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637189+0000", 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011364199999999999 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170186 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1839104~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.191995276, 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073312293000000001, 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170186 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564018+0000", 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.99e-07 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564021+0000", 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7410000000000001e-06 2024-09-12T07:21:44.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564027+0000", 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5119999999999997e-06 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564033+0000", 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4970000000000001e-06 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637230+0000", 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073197613999999994 2024-09-12T07:21:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637271+0000", 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1001000000000003e-05 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637330+0000", 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8328999999999997e-05 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170189 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1921024~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.191660145999997, 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073393609999999998, 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170189 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564353+0000", 2024-09-12T07:21:44.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9500000000000003e-07 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564355+0000", 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1409999999999999e-06 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564362+0000", 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6279999999999999e-06 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564367+0000", 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5520000000000004e-06 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637563+0000", 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073196285 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637641+0000", 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7051000000000005e-05 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637746+0000", 2024-09-12T07:21:44.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010545800000000001 2024-09-12T07:21:44.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.834 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.834 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170191 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1961984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.834 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:44.834 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.191377183999997, 2024-09-12T07:21:44.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073246585000000003, 2024-09-12T07:21:44.834 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170191 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564636+0000", 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6400000000000002e-07 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564639+0000", 2024-09-12T07:21:44.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.399e-06 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564646+0000", 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0999999999999998e-06 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564653+0000", 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9659999999999999e-06 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637785+0000", 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073131357999999994 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637826+0000", 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1742000000000001e-05 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637882+0000", 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5455999999999999e-05 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.839 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170192 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 2002944~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.839 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:44.839 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.19134717, 2024-09-12T07:21:44.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073448428999999996, 2024-09-12T07:21:44.839 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170192 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564666+0000", 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564668+0000", 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564675+0000", 2024-09-12T07:21:44.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2230000000000004e-06 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564680+0000", 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3210000000000004e-06 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637923+0000", 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073243483999999998 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638011+0000", 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7149000000000002e-05 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638114+0000", 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103278 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170198 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 2084864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:44.843 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.190784301000001, 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073259037999999999, 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170198 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:44.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565229+0000", 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0200000000000002e-07 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565231+0000", 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2330000000000001e-06 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565238+0000", 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.596e-06 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565243+0000", 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.31e-06 2024-09-12T07:21:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638324+0000", 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073080720000000002 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638391+0000", 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7037000000000002e-05 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638487+0000", 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6639999999999996e-05 2024-09-12T07:21:44.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.848 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.848 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.848 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:44.848 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:44.848 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:44.849 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:44.850 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:44.851+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:44.950 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:44.952+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:44.958 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:44.958 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:44.958 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:44.958 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:44.967 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:45.051 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:45.052+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:45.075 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:45.075 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:45.075 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:45.075 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:45.075 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:45.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:45.135 INFO:teuthology.orchestra.run.smithi046.stderr:Exporting 3.5 info 3.5( v 140'536 (0'0,140'536] local-lis/les=106/107 n=36 ec=24/24 lis/c=106/106 les/c/f=107/107/0 sis=106) 2024-09-12T07:21:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a0571bb6:::rbd_data.116031710b3d.0000000000000094:head# 2024-09-12T07:21:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a067ff0a:::rbd_data.116052cf9e71.000000000000001b:head# 2024-09-12T07:21:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a1189c58:::rbd_data.1160d8a91ade.00000000000000c1:head# 2024-09-12T07:21:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a159171a:::rbd_data.116031710b3d.0000000000000075:head# 2024-09-12T07:21:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a217c542:::rbd_data.11604a4c57c2.0000000000000054:head# 2024-09-12T07:21:45.143 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a2398328:::rbd_data.116011b0bca6.0000000000000028:head# 2024-09-12T07:21:45.143 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a2663dcb:::rbd_data.116087d6d6a2.0000000000000012:c# 2024-09-12T07:21:45.145 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a2663dcb:::rbd_data.116087d6d6a2.0000000000000012:head# 2024-09-12T07:21:45.145 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a2891995:::rbd_data.116031710b3d.000000000000007c:head# 2024-09-12T07:21:45.145 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a299b40a:::rbd_data.1160d8a91ade.00000000000000ad:head# 2024-09-12T07:21:45.145 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a2c346e7:::rbd_data.11607cf80ea.0000000000000000:head# 2024-09-12T07:21:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a328b86d:::rbd_data.116031710b3d.0000000000000099:head# 2024-09-12T07:21:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a351f3bf:::rbd_data.11604a4c57c2.0000000000000006:head# 2024-09-12T07:21:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a3ce608f:::rbd_data.1160b01e9f21.0000000000000001:12# 2024-09-12T07:21:45.151 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:45.153+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a3ce608f:::rbd_data.1160b01e9f21.0000000000000001:head# 2024-09-12T07:21:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a40c4214:::rbd_data.11607a56846.0000000000000007:head# 2024-09-12T07:21:45.162 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a44b3af2:::rbd_data.116077eca116.000000000000000b:f# 2024-09-12T07:21:45.163 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a44b3af2:::rbd_data.116077eca116.000000000000000b:head# 2024-09-12T07:21:45.163 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a461385b:::rbd_data.1160b01e9f21.0000000000000015:12# 2024-09-12T07:21:45.165 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a461385b:::rbd_data.1160b01e9f21.0000000000000015:head# 2024-09-12T07:21:45.165 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a4a0bff1:::rbd_data.1160d8a91ade.000000000000005f:10# 2024-09-12T07:21:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a4a0bff1:::rbd_data.1160d8a91ade.000000000000005f:head# 2024-09-12T07:21:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a51c63e5:::rbd_id.image_client.0-clone6:head# 2024-09-12T07:21:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a5d6d8dc:::rbd_data.1160b2633c70.00000000000000d2:head# 2024-09-12T07:21:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a5fd1f58:::rbd_data.1160d8a91ade.00000000000000a2:head# 2024-09-12T07:21:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a63e8cc2:::rbd_data.1160b01e9f21.0000000000000005:12# 2024-09-12T07:21:45.174 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a63e8cc2:::rbd_data.1160b01e9f21.0000000000000005:head# 2024-09-12T07:21:45.176 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a64fe51e:::rbd_data.116063d335b3.0000000000000008:8# 2024-09-12T07:21:45.188 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a64fe51e:::rbd_data.116063d335b3.0000000000000008:head# 2024-09-12T07:21:45.189 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a6d9dda1:::rbd_data.1160d8a91ade.00000000000000ba:head# 2024-09-12T07:21:45.189 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a79293fd:::rbd_data.1160b01e9f21.0000000000000013:12# 2024-09-12T07:21:45.192 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a79293fd:::rbd_data.1160b01e9f21.0000000000000013:head# 2024-09-12T07:21:45.192 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a79c08a3:::rbd_data.1160b01e9f21.0000000000000002:12# 2024-09-12T07:21:45.198 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a79c08a3:::rbd_data.1160b01e9f21.0000000000000002:head# 2024-09-12T07:21:45.200 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a7ad670a:::rbd_data.1160d8a91ade.0000000000000094:head# 2024-09-12T07:21:45.201 INFO:teuthology.orchestra.run.smithi046.stderr:Read #3:a7daa15d:::rbd_id.image_client.0-clone3:head# 2024-09-12T07:21:45.201 INFO:teuthology.orchestra.run.smithi046.stderr:Export successful 2024-09-12T07:21:45.252 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:45.254+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:45.261 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 101.878383609, 2024-09-12T07:21:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:21:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:21:45.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 101.815305345, 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:21:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:21:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:21:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:21:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:21:45.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 101.81160026000001, 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:21:45.273 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:21:45.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:21:45.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 97.813253981000003, 2024-09-12T07:21:45.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:45.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:21:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:21:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:21:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:21:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:21:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:21:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:21:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:21:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:21:45.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:21:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116552 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5267456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.793924629000003, 2024-09-12T07:21:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034412801999999999, 2024-09-12T07:21:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116552 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0400000000000002e-07 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467850+0000", 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6529999999999998e-06 2024-09-12T07:21:45.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467856+0000", 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467861+0000", 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502080+0000", 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034218947999999999 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502159+0000", 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9121000000000004e-05 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502260+0000", 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100846 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.289 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116553 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5472256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.793903929999999, 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034609677999999998, 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116553 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467868+0000", 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6800000000000001e-07 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467870+0000", 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.773e-06 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467876+0000", 2024-09-12T07:21:45.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0079999999999999e-06 2024-09-12T07:21:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467880+0000", 2024-09-12T07:21:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-12T07:21:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502304+0000", 2024-09-12T07:21:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423924000000002 2024-09-12T07:21:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502379+0000", 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5010000000000002e-05 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502477+0000", 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7942999999999994e-05 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.294 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116554 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5308416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.793884390000002, 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034779583000000003, 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116554 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:21:45.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467889+0000", 2024-09-12T07:21:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7430000000000001e-06 2024-09-12T07:21:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467895+0000", 2024-09-12T07:21:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8159999999999999e-06 2024-09-12T07:21:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467900+0000", 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502507+0000", 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034607252999999998 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502580+0000", 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3337999999999995e-05 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502667+0000", 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6468e-05 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116555 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5349376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.793865416000003, 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034965729000000001, 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.299 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116555 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1899999999999998e-07 2024-09-12T07:21:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467908+0000", 2024-09-12T07:21:45.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5740000000000001e-06 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467914+0000", 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9939999999999997e-06 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467918+0000", 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.622e-06 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502696+0000", 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034777798999999998 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502780+0000", 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3551e-05 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502872+0000", 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.187e-05 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.303 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.304 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116557 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3506176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.304 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:45.304 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.792771001999995, 2024-09-12T07:21:45.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034335510999999999, 2024-09-12T07:21:45.304 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.304 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.304 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.304 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116557 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:45.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469001+0000", 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469003+0000", 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469012+0000", 2024-09-12T07:21:45.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8030000000000003e-06 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469019+0000", 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6959999999999999e-06 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503153+0000", 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034134096000000003 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503236+0000", 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2886000000000006e-05 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503336+0000", 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9919000000000003e-05 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116558 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3469312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.308 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.792740721000001, 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034519743999999998, 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116558 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469034+0000", 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469042+0000", 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469047+0000", 2024-09-12T07:21:45.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503370+0000", 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034323201999999997 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503454+0000", 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4109000000000004e-05 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503550+0000", 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6009999999999997e-05 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116559 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3551232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.792713888999998, 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034689659999999997, 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116559 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469058+0000", 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469061+0000", 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.497e-06 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469069+0000", 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2579999999999995e-06 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469074+0000", 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1279999999999999e-06 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503582+0000", 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034508351 2024-09-12T07:21:45.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503656+0000", 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3234999999999999e-05 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503747+0000", 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1638999999999995e-05 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.318 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116560 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.318 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:45.318 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.792687161000003, 2024-09-12T07:21:45.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034902912000000001, 2024-09-12T07:21:45.318 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116560 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469085+0000", 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4700000000000001e-07 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469087+0000", 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4760000000000001e-06 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469095+0000", 2024-09-12T07:21:45.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2339999999999994e-06 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469100+0000", 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503796+0000", 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034695411000000002 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503868+0000", 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2032000000000004e-05 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503987+0000", 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119369 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116561 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3592192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.322 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.792663762000004, 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035102170000000002, 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116561 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469110+0000", 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1629999999999999e-06 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469116+0000", 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.251e-06 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469121+0000", 2024-09-12T07:21:45.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8210000000000001e-06 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504017+0000", 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034895772999999998 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504108+0000", 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1027999999999997e-05 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504210+0000", 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101729 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116562 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.792642917999999, 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035285642999999998, 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116562 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:45.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469129+0000", 2024-09-12T07:21:45.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:21:45.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469131+0000", 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1550000000000001e-06 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469138+0000", 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5520000000000001e-06 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469142+0000", 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9099999999999996e-06 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504239+0000", 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.03509669 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504311+0000", 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1650999999999996e-05 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504414+0000", 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103286 2024-09-12T07:21:45.331 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.332 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.332 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116563 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.332 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:45.332 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.792622284999993, 2024-09-12T07:21:45.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035471691, 2024-09-12T07:21:45.332 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116563 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:45.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:45.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:21:45.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469153+0000", 2024-09-12T07:21:45.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.05e-06 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469158+0000", 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.874e-06 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469163+0000", 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8069999999999999e-06 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504444+0000", 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035280793999999997 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504523+0000", 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9036999999999996e-05 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504621+0000", 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7743000000000003e-05 2024-09-12T07:21:45.336 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116564 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5435392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.792601173999998, 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035734657000000003, 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.337 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116564 2024-09-12T07:21:45.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.338 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:45.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:45.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469171+0000", 2024-09-12T07:21:45.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:45.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469172+0000", 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.832e-06 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469179+0000", 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1449999999999996e-06 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469184+0000", 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504721+0000", 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035536890000000002 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504809+0000", 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7229000000000001e-05 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504905+0000", 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6337999999999998e-05 2024-09-12T07:21:45.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116565 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5476352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.792580508, 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035933158, 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116565 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469193+0000", 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.973e-06 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469201+0000", 2024-09-12T07:21:45.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3730000000000004e-06 2024-09-12T07:21:45.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469206+0000", 2024-09-12T07:21:45.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1660000000000002e-06 2024-09-12T07:21:45.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504937+0000", 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035731205000000002 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505035+0000", 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7759999999999999e-05 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505124+0000", 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9258000000000005e-05 2024-09-12T07:21:45.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116566 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5271552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.792557922, 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036116848999999999, 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116566 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469214+0000", 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469216+0000", 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8759999999999999e-06 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469222+0000", 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.173e-06 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469227+0000", 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3070000000000002e-06 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505154+0000", 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035926362000000003 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505239+0000", 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5261999999999998e-05 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505330+0000", 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1436999999999999e-05 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.351 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116567 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5312512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.792537393000003, 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036305875000000001, 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116567 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:45.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:45.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.58e-07 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469242+0000", 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469249+0000", 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5549999999999999e-06 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469254+0000", 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9899999999999997e-06 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505361+0000", 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036106923999999999 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505449+0000", 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8531999999999999e-05 2024-09-12T07:21:45.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505540+0000", 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0805999999999998e-05 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116568 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5353472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.792510719999996, 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036561383000000003, 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116568 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469263+0000", 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7549999999999999e-06 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469269+0000", 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7219999999999996e-06 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469273+0000", 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505631+0000", 2024-09-12T07:21:45.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036357997000000003 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505711+0000", 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9331000000000003e-05 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505822+0000", 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000111208 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.361 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.361 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:45.362 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:45.362 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:45.364+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:45.462 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:45.464+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:45.476 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --force --op remove --pgid 3.5 2024-09-12T07:21:45.513 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:45.513 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:45.514 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:45.514 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:45.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:45.563 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:45.565+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:45.665 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:45.666+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:45.670 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:45.670 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:45.670 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:45.670 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:45.670 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:45.679 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:45.765 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:45.767+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:45.788 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:45.788 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169456 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1527808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.268876851999998, 2024-09-12T07:21:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068013471000000006, 2024-09-12T07:21:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169456 2024-09-12T07:21:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520228+0000", 2024-09-12T07:21:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.63e-07 2024-09-12T07:21:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520230+0000", 2024-09-12T07:21:45.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3420000000000002e-06 2024-09-12T07:21:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520237+0000", 2024-09-12T07:21:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5950000000000004e-06 2024-09-12T07:21:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520242+0000", 2024-09-12T07:21:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:45.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588003+0000", 2024-09-12T07:21:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067761181000000004 2024-09-12T07:21:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588071+0000", 2024-09-12T07:21:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8095000000000004e-05 2024-09-12T07:21:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588241+0000", 2024-09-12T07:21:45.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016951999999999999 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169457 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1568768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.268855066999997, 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394231, 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.856 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169457 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520250+0000", 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6899999999999997e-07 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520252+0000", 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1959999999999998e-06 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520259+0000", 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1509999999999998e-06 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520264+0000", 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588275+0000", 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068010854999999995 2024-09-12T07:21:45.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588350+0000", 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5402999999999996e-05 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588643+0000", 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029306599999999998 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169458 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.268832590999999, 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069257746999999995, 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169458 2024-09-12T07:21:45.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520274+0000", 2024-09-12T07:21:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3350000000000001e-06 2024-09-12T07:21:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520281+0000", 2024-09-12T07:21:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6819999999999997e-06 2024-09-12T07:21:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520286+0000", 2024-09-12T07:21:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0479999999999998e-06 2024-09-12T07:21:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588740+0000", 2024-09-12T07:21:45.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068453571000000005 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588810+0000", 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0611000000000005e-05 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589529+0000", 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071908599999999997 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.865 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169459 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1650688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.268810299000002, 2024-09-12T07:21:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069503178999999998, 2024-09-12T07:21:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169459 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520296+0000", 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1890000000000001e-06 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520303+0000", 2024-09-12T07:21:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6749999999999996e-06 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520308+0000", 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1240000000000004e-06 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589563+0000", 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069254688999999994 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589637+0000", 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3711e-05 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589797+0000", 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016039799999999999 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169460 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1691648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:45.870 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.268789247000001, 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069750706999999995, 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169460 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:45.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7399999999999999e-07 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520318+0000", 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2639999999999998e-06 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520324+0000", 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3260000000000001e-06 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520329+0000", 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6959999999999996e-06 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589830+0000", 2024-09-12T07:21:45.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069501010000000002 2024-09-12T07:21:45.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589903+0000", 2024-09-12T07:21:45.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3558000000000003e-05 2024-09-12T07:21:45.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590066+0000", 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016247899999999999 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169461 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.268768119000001, 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069982339000000005, 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.875 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169461 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:45.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520339+0000", 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520345+0000", 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5359999999999998e-06 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520350+0000", 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.079e-06 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590098+0000", 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069747117999999997 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590168+0000", 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9982999999999997e-05 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590318+0000", 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015076799999999999 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.879 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169462 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1773568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.880 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:45.880 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.268746004, 2024-09-12T07:21:45.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070210122, 2024-09-12T07:21:45.880 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.880 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.880 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.880 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.880 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.880 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169462 2024-09-12T07:21:45.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.881 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:45.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:45.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520359+0000", 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520361+0000", 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0109999999999999e-06 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520367+0000", 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.285e-06 2024-09-12T07:21:45.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520374+0000", 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0899999999999999e-06 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590348+0000", 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069974233999999996 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590418+0000", 2024-09-12T07:21:45.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9314999999999996e-05 2024-09-12T07:21:45.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590568+0000", 2024-09-12T07:21:45.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000150693 2024-09-12T07:21:45.884 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.884 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.884 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169463 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.268722111000002, 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070436171000000006, 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169463 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524036+0000", 2024-09-12T07:21:45.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036543130000000002 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524043+0000", 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0639999999999996e-06 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524058+0000", 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.411e-05 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524065+0000", 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9780000000000005e-06 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590598+0000", 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066532540000000001 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590669+0000", 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0598000000000004e-05 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590818+0000", 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014956799999999999 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169469 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.264904846999997, 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068176724999999994, 2024-09-12T07:21:45.889 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169469 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524200+0000", 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524202+0000", 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5100000000000001e-06 2024-09-12T07:21:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524209+0000", 2024-09-12T07:21:45.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:45.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524213+0000", 2024-09-12T07:21:45.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-12T07:21:45.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592153+0000", 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067939948 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592225+0000", 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2078999999999999e-05 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592376+0000", 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015063000000000001 2024-09-12T07:21:45.893 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.894 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.894 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169470 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2101248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.894 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:45.894 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.264883826000002, 2024-09-12T07:21:45.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394620000000003, 2024-09-12T07:21:45.894 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169470 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524221+0000", 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8700000000000001e-07 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524223+0000", 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1619999999999998e-06 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524230+0000", 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.657e-06 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524235+0000", 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0799999999999996e-06 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592429+0000", 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068194553000000005 2024-09-12T07:21:45.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592512+0000", 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3123999999999999e-05 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592615+0000", 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102657 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169471 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2142208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.264863040999998, 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068565151000000005, 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169471 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6300000000000001e-07 2024-09-12T07:21:45.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524244+0000", 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524250+0000", 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1890000000000004e-06 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524256+0000", 2024-09-12T07:21:45.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5230000000000003e-06 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592647+0000", 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068391788999999995 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592719+0000", 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1859000000000004e-05 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592806+0000", 2024-09-12T07:21:45.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6940000000000004e-05 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169472 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.264841877000002, 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068745155000000002, 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.904 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.904 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169472 2024-09-12T07:21:45.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.904 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:45.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524263+0000", 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4499999999999998e-07 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524265+0000", 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.176e-06 2024-09-12T07:21:45.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524271+0000", 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4439999999999996e-06 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524276+0000", 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7949999999999998e-06 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592836+0000", 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068560461000000003 2024-09-12T07:21:45.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592907+0000", 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0258000000000004e-05 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593007+0000", 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100676 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169473 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2224128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.264821507000001, 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068911932999999995, 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169473 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.908 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:45.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.34e-07 2024-09-12T07:21:45.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524285+0000", 2024-09-12T07:21:45.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0209999999999998e-06 2024-09-12T07:21:45.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524291+0000", 2024-09-12T07:21:45.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8989999999999998e-06 2024-09-12T07:21:45.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524296+0000", 2024-09-12T07:21:45.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0069999999999997e-06 2024-09-12T07:21:45.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593038+0000", 2024-09-12T07:21:45.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068742019000000001 2024-09-12T07:21:45.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593109+0000", 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1075999999999995e-05 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593195+0000", 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5576999999999997e-05 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169474 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2265088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.264801313, 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069079421000000002, 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169474 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:45.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:45.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3700000000000001e-07 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524305+0000", 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1579999999999999e-06 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524311+0000", 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-12T07:21:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524316+0000", 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1109999999999999e-06 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593225+0000", 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068909140999999993 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593298+0000", 2024-09-12T07:21:45.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2432e-05 2024-09-12T07:21:45.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593382+0000", 2024-09-12T07:21:45.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4404e-05 2024-09-12T07:21:45.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.917 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.917 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169475 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2306048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.264781401999997, 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069270256000000002, 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169475 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7500000000000001e-07 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524325+0000", 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1509999999999998e-06 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524332+0000", 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2650000000000002e-06 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524336+0000", 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:21:45.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593418+0000", 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069081988999999996 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593491+0000", 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2619000000000004e-05 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593593+0000", 2024-09-12T07:21:45.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101887 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169476 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.264757322000001, 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069449397999999996, 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169476 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4900000000000001e-07 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524350+0000", 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.481e-06 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524356+0000", 2024-09-12T07:21:45.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6989999999999997e-06 2024-09-12T07:21:45.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524362+0000", 2024-09-12T07:21:45.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2320000000000001e-06 2024-09-12T07:21:45.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593623+0000", 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069261328999999996 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593706+0000", 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2528999999999995e-05 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593796+0000", 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0679000000000004e-05 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169477 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2387968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.264734517000001, 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069660978999999998, 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169477 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.927 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:45.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3299999999999998e-07 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524372+0000", 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1519999999999999e-06 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524379+0000", 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.562e-06 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524383+0000", 2024-09-12T07:21:45.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5430000000000004e-06 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593852+0000", 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069468641999999997 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593926+0000", 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4070000000000001e-05 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594031+0000", 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104677 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169478 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2428928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.264714101000003, 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069830325999999998, 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169478 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524393+0000", 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.092e-06 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524399+0000", 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3539999999999996e-06 2024-09-12T07:21:45.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524404+0000", 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9590000000000003e-06 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594062+0000", 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069658017000000003 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594131+0000", 2024-09-12T07:21:45.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9226000000000004e-05 2024-09-12T07:21:45.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594220+0000", 2024-09-12T07:21:45.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9295999999999993e-05 2024-09-12T07:21:45.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.935 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.936 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169479 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2469888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.936 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:45.936 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.264693627, 2024-09-12T07:21:45.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069996201999999993, 2024-09-12T07:21:45.936 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169479 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:45.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530070+0000", 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056590099999999999 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530075+0000", 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8200000000000002e-06 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530085+0000", 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530093+0000", 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5519999999999999e-06 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594250+0000", 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064157642000000001 2024-09-12T07:21:45.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594318+0000", 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.737e-05 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594407+0000", 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9117999999999996e-05 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169502 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 3411968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 36.258238702, 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068036163999999996, 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169502 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.941 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535987+0000", 2024-09-12T07:21:45.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051212549999999999 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535993+0000", 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0639999999999999e-06 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536003+0000", 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0549e-05 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536012+0000", 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2309999999999996e-06 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598753+0000", 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062741665000000002 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598822+0000", 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8689999999999995e-05 2024-09-12T07:21:45.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:45.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:45.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:45.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598902+0000", 2024-09-12T07:21:45.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9709999999999994e-05 2024-09-12T07:21:45.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.945 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:45.945 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:45.945 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:45.947 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:45.948 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:45.949+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:46.047 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:46.049+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:46.067 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:46.068 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:46.068 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:46.068 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:46.077 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:46.160 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:46.162+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:46.226 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:21:46.229 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:46.229 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:46.229 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:46.229 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:46.229 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:46.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:46.280 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:46.282+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:46.350 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:46.352+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:46.368 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:46.390 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:46.390 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:46.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:46.390 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:46.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.390 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:21:46.390 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:46.390 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 108.776550681, 2024-09-12T07:21:46.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:21:46.390 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.390 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.390 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:21:46.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:21:46.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:21:46.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:21:46.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:21:46.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:21:46.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.393 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.394 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 103.641090425, 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:46.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:21:46.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:21:46.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.397 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.397 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:21:46.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:21:46.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:21:46.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:21:46.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:21:46.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:21:46.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:21:46.399 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.399 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.399 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 103.636863304, 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:46.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:21:46.403 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 102.90911264899999, 2024-09-12T07:21:46.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:46.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:21:46.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:21:46.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:21:46.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:21:46.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 102.82013606, 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.410 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:21:46.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 102.78858473, 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:21:46.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:21:46.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:21:46.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:21:46.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:21:46.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 102.785384194, 2024-09-12T07:21:46.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:21:46.419 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.419 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.419 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.419 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.419 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.419 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:46.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:46.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:21:46.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:21:46.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:21:46.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:21:46.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:21:46.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:21:46.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:46.423 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 102.782138446, 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.424 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:21:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:21:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:21:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:21:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 102.779896615, 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.428 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.429 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:21:46.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.432 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 102.688789411, 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:21:46.434 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.435 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:21:46.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 102.684961323, 2024-09-12T07:21:46.437 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:46.438 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.439 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.440 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:21:46.441 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.703048542000005, 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:21:46.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:21:46.444 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.445 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.446 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.691234898000005, 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.447 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.448 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:21:46.449 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.450 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 79.689898850999995, 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.451 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.452 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.453 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.455 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165942 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 38.189064836999997, 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93685439599999998, 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165942 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200798+0000", 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.455e-06 2024-09-12T07:21:46.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200802+0000", 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2390000000000004e-06 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200814+0000", 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1816e-05 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.458 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200820+0000", 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8760000000000002e-06 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200899+0000", 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8695999999999993e-05 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200912+0000", 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3545e-05 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.459 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137326+0000", 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93641327100000005 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137519+0000", 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000193298 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137651+0000", 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013219999999999999 2024-09-12T07:21:46.460 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165957 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 38.188452959999999, 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93691399799999997, 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:46.461 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165957 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201409+0000", 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1999999999999999e-07 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201412+0000", 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2590000000000001e-06 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.463 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201423+0000", 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0547000000000001e-05 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201428+0000", 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.101e-06 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201488+0000", 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9426999999999999e-05 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.464 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201498+0000", 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0883999999999999e-05 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138057+0000", 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93655902000000002 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138168+0000", 2024-09-12T07:21:46.465 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011034199999999999 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138322+0000", 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015469799999999999 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165958 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 38.188421888000001, 2024-09-12T07:21:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.937070242, 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165958 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.58e-07 2024-09-12T07:21:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201442+0000", 2024-09-12T07:21:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.719e-06 2024-09-12T07:21:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201447+0000", 2024-09-12T07:21:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4770000000000002e-06 2024-09-12T07:21:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201450+0000", 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.219e-06 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201530+0000", 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9833000000000006e-05 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201538+0000", 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.028e-06 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138367+0000", 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93682842899999996 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138429+0000", 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1890000000000005e-05 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138510+0000", 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1289e-05 2024-09-12T07:21:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165973 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 38.188289263000001, 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93711003900000001, 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165973 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201573+0000", 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8599999999999996e-07 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201580+0000", 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3810000000000004e-06 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201590+0000", 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0169999999999999e-05 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201596+0000", 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4110000000000001e-06 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201645+0000", 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9156999999999997e-05 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201656+0000", 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1112e-05 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138538+0000", 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93688176899999998 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138595+0000", 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6759000000000003e-05 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138682+0000", 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7293999999999995e-05 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165974 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 38.188251115, 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93725339100000005, 2024-09-12T07:21:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165974 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201611+0000", 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201614+0000", 2024-09-12T07:21:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.757e-06 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201620+0000", 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.015e-06 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201623+0000", 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6210000000000001e-06 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201679+0000", 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6017999999999998e-05 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201685+0000", 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2560000000000002e-06 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138699+0000", 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93701424600000005 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138749+0000", 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9951000000000003e-05 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138864+0000", 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115017 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-12T07:21:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.894331154, 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0249796689999999, 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495532+0000", 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6130000000000001e-06 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495542+0000", 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7990000000000005e-06 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495556+0000", 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4737e-05 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495567+0000", 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0348e-05 2024-09-12T07:21:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495628+0000", 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0711999999999997e-05 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495638+0000", 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0482999999999999e-05 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520066+0000", 2024-09-12T07:21:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0244278959999999 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520246+0000", 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000179815 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520510+0000", 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026426599999999998 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:46.488 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:46.488 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:46.489 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:46.490 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:46.490+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:46.497 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:46.531 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-12T07:21:46.554 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:46.554 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:46.572 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:46.572 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:46.589 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:46.591+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:46.589 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:46.609 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:46.625 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:46.628 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T07:21:46.629 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T07:21:46.638 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:46.638 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:46.647 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:46.691 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:46.693+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:46.754 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:46.754 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:46.763 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:46.805 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:46.806+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:46.891 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:46.893+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:46.911 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:46.911 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:46.920 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:46.992 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:46.994+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:47.028 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:47.028 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:47.036 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:47.120 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:47.122+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:47.183 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:47.183 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:47.191 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:47.213 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:21:47.232 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:47.234+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:47.294 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:47.296+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: marking collection for removal 2024-09-12T07:21:47.315 INFO:teuthology.orchestra.run.smithi046.stdout:setting '_remove' omap key 2024-09-12T07:21:47.315 INFO:teuthology.orchestra.run.smithi046.stdout:finish_remove_pgs 3.5_head removing 3.5 2024-09-12T07:21:47.326 INFO:teuthology.orchestra.run.smithi046.stdout:Remove successful 2024-09-12T07:21:47.341 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:47.341 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:47.350 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:47.395 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:47.397+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:47.457 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:47.457 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:47.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:47.496 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:47.498+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:47.597 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:47.598+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:47.613 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:47.613 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:47.622 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:47.707 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:47.709+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:47.770 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:47.770 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:47.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:47.819 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:47.821+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:47.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.5.2 2024-09-12T07:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:47.900 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:47.902+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:47.904 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:48.001 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:48.003+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:48.011 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:48.011 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:48.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:48.102 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:48.103+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:48.169 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:48.170 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:48.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:48.203 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:48.205+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:48.300 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:48.301 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:48.304 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:48.305+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:48.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:48.405 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:48.406+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:48.459 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:48.459 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:48.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:48.506 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:48.508+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:48.607 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:48.609+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:48.616 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:48.616 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:48.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:48.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:48.709 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:48.711+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:48.776 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:48.777 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:48.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:48.791 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:48.809 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-12T07:21:48.810 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:48.811+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:48.826 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:48.826 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:48.843 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:48.844 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:48.864 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:48.865 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:48.879 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:48.896 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:48.899 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T07:21:48.900 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T07:21:48.900 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:48.900 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:48.901 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:48.901 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:48.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:48.910 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:48.912+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:49.012 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:49.014+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:49.060 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169456 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1527808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:49.060 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.540572898999997, 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068013471000000006, 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169456 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:49.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520228+0000", 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.63e-07 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520230+0000", 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3420000000000002e-06 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520237+0000", 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5950000000000004e-06 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520242+0000", 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588003+0000", 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067761181000000004 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588071+0000", 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8095000000000004e-05 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588241+0000", 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016951999999999999 2024-09-12T07:21:49.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169457 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1568768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.540551114000003, 2024-09-12T07:21:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394231, 2024-09-12T07:21:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169457 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520250+0000", 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6899999999999997e-07 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520252+0000", 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1959999999999998e-06 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520259+0000", 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1509999999999998e-06 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520264+0000", 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588275+0000", 2024-09-12T07:21:49.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068010854999999995 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588350+0000", 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5402999999999996e-05 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588643+0000", 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029306599999999998 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.070 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169458 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.540528637999998, 2024-09-12T07:21:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069257746999999995, 2024-09-12T07:21:49.072 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169458 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520274+0000", 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3350000000000001e-06 2024-09-12T07:21:49.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520281+0000", 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6819999999999997e-06 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520286+0000", 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0479999999999998e-06 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588740+0000", 2024-09-12T07:21:49.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068453571000000005 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588810+0000", 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0611000000000005e-05 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589529+0000", 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071908599999999997 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169459 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1650688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.540506346000001, 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069503178999999998, 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169459 2024-09-12T07:21:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520296+0000", 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1890000000000001e-06 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520303+0000", 2024-09-12T07:21:49.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6749999999999996e-06 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520308+0000", 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1240000000000004e-06 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589563+0000", 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069254688999999994 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589637+0000", 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3711e-05 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589797+0000", 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016039799999999999 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169460 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1691648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.081 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.540485294, 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069750706999999995, 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169460 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7399999999999999e-07 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520318+0000", 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2639999999999998e-06 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520324+0000", 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3260000000000001e-06 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520329+0000", 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6959999999999996e-06 2024-09-12T07:21:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589830+0000", 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069501010000000002 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589903+0000", 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3558000000000003e-05 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590066+0000", 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016247899999999999 2024-09-12T07:21:49.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169461 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.540464166, 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069982339000000005, 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169461 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520339+0000", 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520345+0000", 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5359999999999998e-06 2024-09-12T07:21:49.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520350+0000", 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.079e-06 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590098+0000", 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069747117999999997 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590168+0000", 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9982999999999997e-05 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590318+0000", 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015076799999999999 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169462 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1773568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:49.090 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.540442050999999, 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070210122, 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169462 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520359+0000", 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520361+0000", 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0109999999999999e-06 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520367+0000", 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.285e-06 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520374+0000", 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0899999999999999e-06 2024-09-12T07:21:49.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590348+0000", 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069974233999999996 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590418+0000", 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9314999999999996e-05 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590568+0000", 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000150693 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169463 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.540418158000001, 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070436171000000006, 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169463 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524036+0000", 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036543130000000002 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524043+0000", 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0639999999999996e-06 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524058+0000", 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.411e-05 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524065+0000", 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9780000000000005e-06 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590598+0000", 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066532540000000001 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590669+0000", 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0598000000000004e-05 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590818+0000", 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014956799999999999 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.099 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169469 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.536600894000003, 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068176724999999994, 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169469 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524200+0000", 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524202+0000", 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5100000000000001e-06 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524209+0000", 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524213+0000", 2024-09-12T07:21:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592153+0000", 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067939948 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592225+0000", 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2078999999999999e-05 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592376+0000", 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015063000000000001 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169470 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2101248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.536579873000001, 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394620000000003, 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169470 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524221+0000", 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8700000000000001e-07 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524223+0000", 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1619999999999998e-06 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524230+0000", 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.657e-06 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524235+0000", 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0799999999999996e-06 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592429+0000", 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068194553000000005 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592512+0000", 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3123999999999999e-05 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592615+0000", 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102657 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169471 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2142208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.536559087999997, 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068565151000000005, 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169471 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:49.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6300000000000001e-07 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524244+0000", 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524250+0000", 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1890000000000004e-06 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524256+0000", 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5230000000000003e-06 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592647+0000", 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068391788999999995 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592719+0000", 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1859000000000004e-05 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592806+0000", 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6940000000000004e-05 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169472 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.536537924000001, 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068745155000000002, 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169472 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524263+0000", 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4499999999999998e-07 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524265+0000", 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.176e-06 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524271+0000", 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4439999999999996e-06 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524276+0000", 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7949999999999998e-06 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592836+0000", 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068560461000000003 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592907+0000", 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0258000000000004e-05 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593007+0000", 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100676 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169473 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2224128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.117 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.536517554, 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068911932999999995, 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169473 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.34e-07 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524285+0000", 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0209999999999998e-06 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524291+0000", 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8989999999999998e-06 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524296+0000", 2024-09-12T07:21:49.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0069999999999997e-06 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593038+0000", 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068742019000000001 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593109+0000", 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1075999999999995e-05 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593195+0000", 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5576999999999997e-05 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169474 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2265088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.536497359999998, 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069079421000000002, 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169474 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:49.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3700000000000001e-07 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524305+0000", 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1579999999999999e-06 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524311+0000", 2024-09-12T07:21:49.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524316+0000", 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1109999999999999e-06 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593225+0000", 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068909140999999993 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593298+0000", 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2432e-05 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593382+0000", 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4404e-05 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169475 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2306048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.126 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.536477449000003, 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069270256000000002, 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169475 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:49.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7500000000000001e-07 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524325+0000", 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1509999999999998e-06 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524332+0000", 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2650000000000002e-06 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524336+0000", 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593418+0000", 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069081988999999996 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593491+0000", 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2619000000000004e-05 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593593+0000", 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101887 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.130 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169476 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.536453369, 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069449397999999996, 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169476 2024-09-12T07:21:49.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4900000000000001e-07 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524350+0000", 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.481e-06 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524356+0000", 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6989999999999997e-06 2024-09-12T07:21:49.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524362+0000", 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2320000000000001e-06 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593623+0000", 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069261328999999996 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593706+0000", 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2528999999999995e-05 2024-09-12T07:21:49.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593796+0000", 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0679000000000004e-05 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169477 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2387968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.536430564, 2024-09-12T07:21:49.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069660978999999998, 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169477 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:49.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3299999999999998e-07 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524372+0000", 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1519999999999999e-06 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524379+0000", 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.562e-06 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524383+0000", 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5430000000000004e-06 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593852+0000", 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069468641999999997 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593926+0000", 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4070000000000001e-05 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594031+0000", 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104677 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.139 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169478 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2428928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.536410148000002, 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069830325999999998, 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169478 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524393+0000", 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.092e-06 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524399+0000", 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3539999999999996e-06 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524404+0000", 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9590000000000003e-06 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594062+0000", 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069658017000000003 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594131+0000", 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9226000000000004e-05 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594220+0000", 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9295999999999993e-05 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169479 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2469888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.536389673999999, 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069996201999999993, 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169479 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530070+0000", 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056590099999999999 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530075+0000", 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8200000000000002e-06 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530085+0000", 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530093+0000", 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5519999999999999e-06 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594250+0000", 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064157642000000001 2024-09-12T07:21:49.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594318+0000", 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.737e-05 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594407+0000", 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9117999999999996e-05 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169502 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 3411968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.529934748999999, 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068036163999999996, 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169502 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:49.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535987+0000", 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051212549999999999 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535993+0000", 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0639999999999999e-06 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536003+0000", 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0549e-05 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536012+0000", 2024-09-12T07:21:49.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2309999999999996e-06 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598753+0000", 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062741665000000002 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598822+0000", 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8689999999999995e-05 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:49.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598902+0000", 2024-09-12T07:21:49.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9709999999999994e-05 2024-09-12T07:21:49.153 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.153 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.153 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.153 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:49.153 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:49.153 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:49.154 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:49.155 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:49.156+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:49.234 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:21:49.265 INFO:teuthology.orchestra.run.smithi046.stdout:get_pg_num_history pg_num_history pg_num_history(e140 pg_nums {1={16=1},2={19=8},3={24=32}} deleted_pools ) 2024-09-12T07:21:49.266 INFO:teuthology.orchestra.run.smithi046.stdout:Importing pgid 3.5 2024-09-12T07:21:49.266 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:49.267+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:49.267 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a0571bb6:::rbd_data.116031710b3d.0000000000000094:head# 2024-09-12T07:21:49.267 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 18=[]:{} 2024-09-12T07:21:49.267 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a067ff0a:::rbd_data.116052cf9e71.000000000000001b:head# 2024-09-12T07:21:49.267 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 9=[]:{} 2024-09-12T07:21:49.267 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a1189c58:::rbd_data.1160d8a91ade.00000000000000c1:head# 2024-09-12T07:21:49.267 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 10=[]:{} 2024-09-12T07:21:49.268 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a159171a:::rbd_data.116031710b3d.0000000000000075:head# 2024-09-12T07:21:49.268 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 18=[]:{} 2024-09-12T07:21:49.268 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a217c542:::rbd_data.11604a4c57c2.0000000000000054:head# 2024-09-12T07:21:49.268 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 0=[]:{} 2024-09-12T07:21:49.270 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a2398328:::rbd_data.116011b0bca6.0000000000000028:head# 2024-09-12T07:21:49.270 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 0=[]:{} 2024-09-12T07:21:49.270 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a2663dcb:::rbd_data.116087d6d6a2.0000000000000012:c# 2024-09-12T07:21:49.272 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a2663dcb:::rbd_data.116087d6d6a2.0000000000000012:head# 2024-09-12T07:21:49.272 INFO:teuthology.orchestra.run.smithi046.stdout:snapset c=[]:{c=[c]} 2024-09-12T07:21:49.272 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a2891995:::rbd_data.116031710b3d.000000000000007c:head# 2024-09-12T07:21:49.272 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 18=[]:{} 2024-09-12T07:21:49.272 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a299b40a:::rbd_data.1160d8a91ade.00000000000000ad:head# 2024-09-12T07:21:49.273 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 10=[]:{} 2024-09-12T07:21:49.273 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a2c346e7:::rbd_data.11607cf80ea.0000000000000000:head# 2024-09-12T07:21:49.273 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 0=[]:{} 2024-09-12T07:21:49.278 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a328b86d:::rbd_data.116031710b3d.0000000000000099:head# 2024-09-12T07:21:49.278 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 18=[]:{} 2024-09-12T07:21:49.278 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a351f3bf:::rbd_data.11604a4c57c2.0000000000000006:head# 2024-09-12T07:21:49.278 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 0=[]:{} 2024-09-12T07:21:49.278 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a3ce608f:::rbd_data.1160b01e9f21.0000000000000001:12# 2024-09-12T07:21:49.281 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:49.282 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:49.282 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:49.282 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:49.285 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a3ce608f:::rbd_data.1160b01e9f21.0000000000000001:head# 2024-09-12T07:21:49.285 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 12=[]:{12=[12]} 2024-09-12T07:21:49.286 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a40c4214:::rbd_data.11607a56846.0000000000000007:head# 2024-09-12T07:21:49.287 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 0=[]:{} 2024-09-12T07:21:49.292 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:49.292 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a44b3af2:::rbd_data.116077eca116.000000000000000b:f# 2024-09-12T07:21:49.293 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a44b3af2:::rbd_data.116077eca116.000000000000000b:head# 2024-09-12T07:21:49.293 INFO:teuthology.orchestra.run.smithi046.stdout:snapset f=[]:{f=[f]} 2024-09-12T07:21:49.293 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a461385b:::rbd_data.1160b01e9f21.0000000000000015:12# 2024-09-12T07:21:49.299 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a461385b:::rbd_data.1160b01e9f21.0000000000000015:head# 2024-09-12T07:21:49.299 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 12=[]:{12=[12]} 2024-09-12T07:21:49.299 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a4a0bff1:::rbd_data.1160d8a91ade.000000000000005f:10# 2024-09-12T07:21:49.300 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a4a0bff1:::rbd_data.1160d8a91ade.000000000000005f:head# 2024-09-12T07:21:49.300 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 10=[]:{10=[10]} 2024-09-12T07:21:49.301 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a51c63e5:::rbd_id.image_client.0-clone6:head# 2024-09-12T07:21:49.301 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 0=[]:{} 2024-09-12T07:21:49.301 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a5d6d8dc:::rbd_data.1160b2633c70.00000000000000d2:head# 2024-09-12T07:21:49.301 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 0=[]:{} 2024-09-12T07:21:49.302 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a5fd1f58:::rbd_data.1160d8a91ade.00000000000000a2:head# 2024-09-12T07:21:49.302 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 10=[]:{} 2024-09-12T07:21:49.302 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a63e8cc2:::rbd_data.1160b01e9f21.0000000000000005:12# 2024-09-12T07:21:49.308 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a63e8cc2:::rbd_data.1160b01e9f21.0000000000000005:head# 2024-09-12T07:21:49.308 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 12=[]:{12=[12]} 2024-09-12T07:21:49.309 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a64fe51e:::rbd_data.116063d335b3.0000000000000008:8# 2024-09-12T07:21:49.333 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a64fe51e:::rbd_data.116063d335b3.0000000000000008:head# 2024-09-12T07:21:49.333 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 8=[]:{8=[8]} 2024-09-12T07:21:49.334 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a6d9dda1:::rbd_data.1160d8a91ade.00000000000000ba:head# 2024-09-12T07:21:49.334 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 10=[]:{} 2024-09-12T07:21:49.335 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a79293fd:::rbd_data.1160b01e9f21.0000000000000013:12# 2024-09-12T07:21:49.339 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a79293fd:::rbd_data.1160b01e9f21.0000000000000013:head# 2024-09-12T07:21:49.340 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 12=[]:{12=[12]} 2024-09-12T07:21:49.340 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a79c08a3:::rbd_data.1160b01e9f21.0000000000000002:12# 2024-09-12T07:21:49.346 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a79c08a3:::rbd_data.1160b01e9f21.0000000000000002:head# 2024-09-12T07:21:49.346 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 12=[]:{12=[12]} 2024-09-12T07:21:49.347 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a7ad670a:::rbd_data.1160d8a91ade.0000000000000094:head# 2024-09-12T07:21:49.347 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 10=[]:{} 2024-09-12T07:21:49.348 INFO:teuthology.orchestra.run.smithi046.stdout:Write #3:a7daa15d:::rbd_id.image_client.0-clone3:head# 2024-09-12T07:21:49.348 INFO:teuthology.orchestra.run.smithi046.stdout:snapset 0=[]:{} 2024-09-12T07:21:49.349 INFO:teuthology.orchestra.run.smithi046.stdout:write_pg epoch 140 info 3.5( v 140'536 (0'0,140'536] local-lis/les=106/107 n=36 ec=24/24 lis/c=106/106 les/c/f=107/107/0 sis=106) 2024-09-12T07:21:49.352 INFO:teuthology.orchestra.run.smithi046.stdout:Import successful 2024-09-12T07:21:49.356 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:49.357+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:49.402 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:49.402 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:49.402 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:49.402 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:49.402 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:49.410 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:49.457 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:49.459+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:49.558 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:49.559+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:49.560 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:49.560 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:49.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:49.560 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:49.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.560 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:21:49.560 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:49.560 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.947461253, 2024-09-12T07:21:49.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:21:49.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:21:49.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.562 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:21:49.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:21:49.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.562 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:21:49.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:21:49.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:21:49.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:21:49.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:21:49.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:21:49.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:21:49.565 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.565 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.565 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 106.812000997, 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:49.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:21:49.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:21:49.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 106.807773876, 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:49.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:21:49.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:21:49.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:49.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:21:49.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:21:49.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:21:49.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:21:49.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:21:49.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:21:49.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:21:49.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:21:49.575 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.575 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.575 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 106.080023221, 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:49.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.99104663200001, 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:21:49.580 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:21:49.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.584 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.585 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.585 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:21:49.585 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:49.585 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.95949530199999, 2024-09-12T07:21:49.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:21:49.585 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.585 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.586 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:21:49.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.589 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.956294766, 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:21:49.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:21:49.594 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.953049018, 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:21:49.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.597 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.950807187, 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:21:49.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.603 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.859699983, 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:49.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:21:49.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:21:49.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:21:49.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:21:49.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:21:49.608 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 105.85587189500001, 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.609 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:21:49.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.873959114000002, 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.613 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.615 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:21:49.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.862145470000002, 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:21:49.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:21:49.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:21:49.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:21:49.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:21:49.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:21:49.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 82.860809423000006, 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.622 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:21:49.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.626 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165942 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.359975409, 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93685439599999998, 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165942 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.628 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200798+0000", 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.455e-06 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200802+0000", 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2390000000000004e-06 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200814+0000", 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1816e-05 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200820+0000", 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8760000000000002e-06 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200899+0000", 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8695999999999993e-05 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200912+0000", 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3545e-05 2024-09-12T07:21:49.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137326+0000", 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93641327100000005 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137519+0000", 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000193298 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137651+0000", 2024-09-12T07:21:49.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013219999999999999 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165957 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.359363532000003, 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93691399799999997, 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:49.632 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165957 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201409+0000", 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1999999999999999e-07 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201412+0000", 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2590000000000001e-06 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.634 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201423+0000", 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0547000000000001e-05 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201428+0000", 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.101e-06 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201488+0000", 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9426999999999999e-05 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201498+0000", 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0883999999999999e-05 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138057+0000", 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93655902000000002 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138168+0000", 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011034199999999999 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138322+0000", 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015469799999999999 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165958 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.359332459999997, 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.937070242, 2024-09-12T07:21:49.637 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165958 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.58e-07 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201442+0000", 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.719e-06 2024-09-12T07:21:49.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201447+0000", 2024-09-12T07:21:49.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4770000000000002e-06 2024-09-12T07:21:49.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201450+0000", 2024-09-12T07:21:49.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.219e-06 2024-09-12T07:21:49.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201530+0000", 2024-09-12T07:21:49.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9833000000000006e-05 2024-09-12T07:21:49.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:49.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201538+0000", 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.028e-06 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138367+0000", 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93682842899999996 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138429+0000", 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1890000000000005e-05 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138510+0000", 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1289e-05 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165973 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.359199834999998, 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93711003900000001, 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.643 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165973 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201573+0000", 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8599999999999996e-07 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201580+0000", 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3810000000000004e-06 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201590+0000", 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0169999999999999e-05 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201596+0000", 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4110000000000001e-06 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201645+0000", 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9156999999999997e-05 2024-09-12T07:21:49.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201656+0000", 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1112e-05 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138538+0000", 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93688176899999998 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138595+0000", 2024-09-12T07:21:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6759000000000003e-05 2024-09-12T07:21:49.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.648 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138682+0000", 2024-09-12T07:21:49.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7293999999999995e-05 2024-09-12T07:21:49.648 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.648 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.648 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.648 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165974 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:49.648 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.359161686999997, 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93725339100000005, 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165974 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.649 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201611+0000", 2024-09-12T07:21:49.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:21:49.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.651 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201614+0000", 2024-09-12T07:21:49.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.757e-06 2024-09-12T07:21:49.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.651 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201620+0000", 2024-09-12T07:21:49.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.015e-06 2024-09-12T07:21:49.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201623+0000", 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6210000000000001e-06 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201679+0000", 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6017999999999998e-05 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201685+0000", 2024-09-12T07:21:49.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2560000000000002e-06 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138699+0000", 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93701424600000005 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138749+0000", 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9951000000000003e-05 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138864+0000", 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115017 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 27.065241726, 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0249796689999999, 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:49.654 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:49.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495532+0000", 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6130000000000001e-06 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495542+0000", 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7990000000000005e-06 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:49.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495556+0000", 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4737e-05 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495567+0000", 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0348e-05 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495628+0000", 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0711999999999997e-05 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495638+0000", 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0482999999999999e-05 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520066+0000", 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0244278959999999 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520246+0000", 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000179815 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:49.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:49.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:49.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520510+0000", 2024-09-12T07:21:49.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026426599999999998 2024-09-12T07:21:49.659 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.659 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.659 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.659 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:49.659 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:49.659 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:49.661 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:49.661 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:49.662+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:49.761 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:49.763+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:49.808 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:49.808 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:49.808 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:49.808 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:49.817 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:49.862 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:49.864+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:49.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /var/log/ceph/exp.3.5.2 2024-09-12T07:21:49.924 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:49.924 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:49.924 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:49.924 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:49.924 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:49.932 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:49.973 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:49.975+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:50.065 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:50.066+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:50.081 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:50.081 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:50.081 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:50.081 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:50.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.081 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.081 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:50.081 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.600288998, 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:50.082 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.083 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:21:50.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:21:50.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.084 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:21:50.084 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:21:50.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.084 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.086 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.086 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:21:50.086 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:21:50.086 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.086 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.086 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.087 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.087 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:50.087 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.600262118, 2024-09-12T07:21:50.087 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:21:50.087 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.087 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.087 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.087 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.087 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.087 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.088 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.089 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:21:50.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.60023809800001, 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:21:50.091 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:21:50.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:21:50.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.600210452, 2024-09-12T07:21:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:21:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.60018699, 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:21:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.600164463, 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:21:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.60014266, 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:21:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.113 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.59350888, 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:21:50.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:21:50.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.59348887199999, 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.118 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.58700584, 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:21:50.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:21:50.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.586970553, 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.127 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:50.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.131 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.58694792, 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.58689386499999, 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:21:50.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.586874924, 2024-09-12T07:21:50.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:21:50.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:21:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.586862987, 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:21:50.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:21:50.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:21:50.149 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.149 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.149 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.149 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.149 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:50.149 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.586851372, 2024-09-12T07:21:50.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:21:50.150 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.150 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.150 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.150 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.150 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.150 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.150 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:21:50.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.150 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:50.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:21:50.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:21:50.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.579838764, 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:21:50.154 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:21:50.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:21:50.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:21:50.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:21:50.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:21:50.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:21:50.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.579815049, 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.159 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:21:50.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.163 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.579684192, 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:21:50.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 106.61601340199999, 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.168 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:21:50.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:21:50.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:50.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.173 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:50.175 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:50.175 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:50.176+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:50.223 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:21:50.275 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:50.276+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:50.288 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:50.288 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:50.288 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:50.289 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:50.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:50.382 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:50.384+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:50.446 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:50.446 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:50.447 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:50.447 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:50.447 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:50.454 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:50.495 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:50.497+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:50.578 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:50.579+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:50.603 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:50.603 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:50.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:50.603 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:50.603 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.603 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:21:50.603 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:50.603 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 118.07432617400001, 2024-09-12T07:21:50.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:21:50.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.608 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.222528508, 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.609 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:21:50.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:21:50.611 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:21:50.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.612 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.612 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:21:50.612 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:21:50.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.613 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.613 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:21:50.613 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:21:50.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.614 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.614 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:21:50.614 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:21:50.614 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.614 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.614 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.615 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.615 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.615 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:50.615 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.222405231, 2024-09-12T07:21:50.615 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:21:50.615 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.616 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.616 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.616 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.616 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.616 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.616 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:21:50.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.617 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.617 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:50.617 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.617 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:50.618 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.618 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.618 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:21:50.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:21:50.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.619 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.619 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:21:50.619 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:21:50.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.620 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.620 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:21:50.620 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:21:50.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.621 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:21:50.621 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:21:50.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.621 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.621 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:21:50.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:21:50.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.622 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.622 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.622 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:21:50.622 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:21:50.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.22220184, 2024-09-12T07:21:50.623 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:50.624 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.625 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:21:50.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:21:50.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:21:50.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.627 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:21:50.627 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:21:50.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.627 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.628 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:21:50.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:21:50.628 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.628 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.628 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.628 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.628 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:50.628 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.222066013, 2024-09-12T07:21:50.628 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:21:50.628 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.629 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.629 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.629 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.629 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.629 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.629 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:21:50.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.629 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.629 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:21:50.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:21:50.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:21:50.631 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:21:50.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.631 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.631 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.221739357, 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.634 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.634 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.634 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.634 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.634 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:21:50.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.634 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.634 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:50.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:50.635 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.635 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.635 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:21:50.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:21:50.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.637 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.637 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:21:50.637 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:21:50.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:21:50.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:21:50.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.638 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.638 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:21:50.638 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:21:50.638 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.638 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.638 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.639 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.639 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:50.639 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.221700469, 2024-09-12T07:21:50.639 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:21:50.639 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.639 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.639 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.640 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.640 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.640 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.640 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:21:50.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.640 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.640 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.640 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:50.640 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:50.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.641 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.641 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:21:50.641 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.642 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:21:50.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.221333595, 2024-09-12T07:21:50.644 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:21:50.646 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:21:50.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.647 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:21:50.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:21:50.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.647 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.647 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:21:50.647 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.648 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.649 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:21:50.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:21:50.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.649 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.649 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.649 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:50.649 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.221198739, 2024-09-12T07:21:50.649 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:21:50.649 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.650 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:21:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:21:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:21:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.652 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:21:50.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:21:50.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:21:50.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:21:50.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.653 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.653 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.653 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:21:50.653 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:21:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.220828848, 2024-09-12T07:21:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:21:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.655 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.655 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.655 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.655 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:21:50.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.655 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.655 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.655 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:50.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:50.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.656 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.656 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.656 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:21:50.656 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:21:50.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:21:50.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:21:50.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.657 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.657 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:21:50.657 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:21:50.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:21:50.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:21:50.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.658 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:21:50.658 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:21:50.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.658 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:21:50.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:21:50.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.659 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.659 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:21:50.659 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:21:50.659 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.659 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.659 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.22063865, 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.660 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:21:50.661 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:21:50.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.662 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.662 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:21:50.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:21:50.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.662 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.662 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:21:50.662 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:21:50.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.663 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:50.664 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.220418571, 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.665 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.666 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:50.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:21:50.668 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.220274906, 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.669 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.670 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:50.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.671 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:21:50.671 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:21:50.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.671 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.671 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:21:50.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:21:50.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:21:50.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:21:50.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.672 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.672 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:21:50.672 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.673 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.220048734, 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.674 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.675 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.676 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:21:50.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.678 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.219848376, 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.679 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:21:50.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:21:50.681 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:21:50.682 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:21:50.683 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.683 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.683 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.683 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.683 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.683 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:50.683 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.21944049299999, 2024-09-12T07:21:50.683 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:21:50.683 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.684 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.684 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.684 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.684 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:21:50.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.684 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.684 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.684 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.684 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:50.684 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:50.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.685 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.685 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:21:50.685 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:21:50.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:21:50.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:21:50.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.686 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.686 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:21:50.686 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:21:50.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:21:50.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:21:50.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.687 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:21:50.687 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:21:50.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.687 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:21:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:21:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:21:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:21:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.21897461099999, 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.691 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:21:50.691 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:21:50.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.691 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:21:50.692 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:21:50.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.692 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.692 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:21:50.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:21:50.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.693 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.693 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:21:50.693 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:21:50.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.694 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.694 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:21:50.694 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:21:50.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.695 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.695 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:21:50.695 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:21:50.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.695 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.696 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.696 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:50.696 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.21737240100001, 2024-09-12T07:21:50.696 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:21:50.696 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.697 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:21:50.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:21:50.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.699 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:21:50.699 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:21:50.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.699 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:21:50.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:21:50.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.700 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.700 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:21:50.700 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:21:50.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 111.216838366, 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.701 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.702 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:21:50.704 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.96064029599999, 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:21:50.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:21:50.707 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.708 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:50.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:50.710 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:50.710 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:21:50.710 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:21:50.710 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.710 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.710 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.710 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:50.710 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:50.710 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:50.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:50.712 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:50.713+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:50.812 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:50.814+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:50.859 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:50.860 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:50.860 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:50.860 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:50.869 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:50.901 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:50.947 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:50.948+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:50.976 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:50.977 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:50.977 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:50.977 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:50.977 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:50.985 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:51.041 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:51.041 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:51.041 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:51.042+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:51.059 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-12T07:21:51.075 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:51.076 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:51.093 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:51.094 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:51.097 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:51.097 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:51.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:51.097 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:51.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.097 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.097 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:51.097 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.546864477, 2024-09-12T07:21:51.097 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:21:51.097 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.097 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.097 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.098 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.098 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.098 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.098 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:21:51.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.098 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.098 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.098 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:51.098 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:21:51.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.100 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:21:51.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:21:51.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:21:51.101 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:21:51.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.101 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.101 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.54684394, 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.102 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.103 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.103 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.103 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.103 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:21:51.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.104 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:51.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.105 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.105 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:51.105 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.106 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:21:51.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:21:51.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:21:51.107 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:21:51.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.107 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.107 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:21:51.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:21:51.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:21:51.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:21:51.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.108 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.108 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:21:51.108 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:21:51.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.109 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.109 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.109 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.109 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:51.109 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.546823937, 2024-09-12T07:21:51.110 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:21:51.110 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.110 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.110 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.110 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.110 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.110 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.110 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:21:51.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.110 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:51.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.111 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.111 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:51.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:21:51.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.114 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.54680398799999, 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:51.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:51.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.116 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.116 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:51.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:21:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:21:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:21:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:21:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:21:51.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:21:51.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.54678312, 2024-09-12T07:21:51.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:21:51.120 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.120 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.120 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.120 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.120 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.120 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.120 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:21:51.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.120 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.120 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:21:51.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.54676141900001, 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.124 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.126 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.127 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.128 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.54674097900001, 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:51.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:21:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:21:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.546713258, 2024-09-12T07:21:51.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:21:51.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:21:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:21:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:21:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:21:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:21:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:21:51.137 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.54669032300001, 2024-09-12T07:21:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:21:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.138 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:21:51.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.142 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.546670023, 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:21:51.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:51.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.147 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.546649747, 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:51.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.150 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:21:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:21:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:21:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:21:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:21:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:21:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:21:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:21:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.546629588, 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:21:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:21:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:21:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:21:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.157 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.54649239, 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:21:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:21:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:21:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:21:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:21:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:21:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:21:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:21:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:21:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.546469742, 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:21:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:21:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:21:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:21:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:21:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:21:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:21:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:21:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.54000775599999, 2024-09-12T07:21:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:21:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:21:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:21:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:21:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:21:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:21:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:21:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.539778119, 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:21:51.176 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.53285442799999, 2024-09-12T07:21:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:21:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.177 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:21:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:21:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:21:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:21:51.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:21:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:21:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:21:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:21:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.532494845, 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:21:51.182 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:21:51.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:21:51.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.186 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 107.532414859, 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:21:51.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:21:51.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:51.191 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 29.603067608, 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0005192119999999, 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:51.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492882+0000", 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492888+0000", 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492904+0000", 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5628e-05 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492912+0000", 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4039999999999995e-06 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492962+0000", 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9425999999999997e-05 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492969+0000", 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1729999999999998e-06 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.493388+0000", 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0004188510000001 2024-09-12T07:21:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:51.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:51.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:51.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.493399+0000", 2024-09-12T07:21:51.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1571e-05 2024-09-12T07:21:51.196 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.196 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.196 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.196 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:51.196 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:51.196 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:51.197 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:51.198 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:51.199 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:51.199 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:51.200 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T07:21:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:51.200 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T07:21:51.202 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:51.199+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:51.300 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:51.300+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:51.332 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:51.333 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:51.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:51.401 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:51.403+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:51.493 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:51.493 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:51.500 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:51.502+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:51.502 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:51.602 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:51.603+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:51.610 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:51.610 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:51.619 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:51.703 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:51.705+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:51.768 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:51.768 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:51.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:51.804 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:51.806+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:51.905 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:51.906+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:51.926 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:51.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:52.005 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:52.007+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:52.083 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:52.084 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:52.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:52.106 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:52.108+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:52.201 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:52.201 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:52.207 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:52.209+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:52.209 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:52.246 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:21:52.308 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:52.310+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:52.370 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:52.370 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:52.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:52.409 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:52.410+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:52.509 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:52.511+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:52.529 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:52.529 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:52.539 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:52.610 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:52.612+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:52.647 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:52.647 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:52.655 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:52.739 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:52.741+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:52.802 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:52.803 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:52.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:52.812 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:52.814+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:52.913 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:52.915+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:52.957 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:52.958 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:52.966 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:53.050 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:53.052+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:53.115 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:53.115 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:53.115 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:53.117+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:53.123 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:53.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:53.241 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:21:53.241 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:53.242+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:53.278 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:53.278 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:53.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:53.316 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:53.317+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:53.362 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:53.379 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:53.379 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:53.380 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-12T07:21:53.395 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:53.395 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:53.400 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:53.400 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:53.409 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:53.412 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:53.412 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:53.417 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:53.418+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:53.430 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:53.430 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:53.447 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:53.463 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:53.464 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:53.467 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T07:21:53.467 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T07:21:53.517 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:53.517 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:53.517 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:53.518 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:53.519+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:53.526 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:53.619 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:53.620+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:53.676 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:53.676 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:53.676 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:53.676 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:53.676 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:53.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:53.720 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:53.722+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:53.822 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:53.823+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:53.833 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:53.833 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:53.833 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:53.833 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:53.834 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.834 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:21:53.834 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:53.834 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 121.304060777, 2024-09-12T07:21:53.834 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:21:53.834 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.834 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.834 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.834 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.834 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:21:53.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:21:53.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.836 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.836 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.836 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:21:53.836 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.837 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:21:53.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:21:53.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:53.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:21:53.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:21:53.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.838 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.838 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.838 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:21:53.838 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:21:53.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.839 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.840 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.840 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:53.840 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.45226311099999, 2024-09-12T07:21:53.840 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:21:53.840 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.840 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.840 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.841 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:21:53.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.843 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.844 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.45213983399999, 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.845 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:53.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:21:53.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.846 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.846 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.846 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:21:53.846 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:21:53.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.847 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:21:53.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:21:53.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:21:53.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:21:53.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.848 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.848 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.848 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:21:53.848 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:21:53.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.849 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.849 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:21:53.849 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:21:53.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.849 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.849 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.849 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:53.849 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.45193644299999, 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.850 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:53.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:21:53.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.851 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.851 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:21:53.851 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:21:53.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.852 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:21:53.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:21:53.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:21:53.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:21:53.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.853 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.853 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:21:53.853 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.451800616, 2024-09-12T07:21:53.855 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:21:53.855 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.855 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.855 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.855 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.855 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.855 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.855 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:21:53.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:21:53.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.857 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.857 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:21:53.857 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:21:53.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.858 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:21:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:21:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:21:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:21:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:21:53.859 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.45147396, 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:53.861 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.862 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.863 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.864 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.451435072, 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.865 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:53.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.866 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.866 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.866 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:21:53.866 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:21:53.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:21:53.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.867 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.867 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:21:53.867 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:21:53.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.867 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:21:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:21:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:21:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:21:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:21:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:21:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:21:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:21:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.451068198, 2024-09-12T07:21:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:21:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:21:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:21:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:21:53.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:21:53.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.873 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:21:53.873 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:21:53.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.873 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.873 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:21:53.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:21:53.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:21:53.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:53.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.874 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.874 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.874 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:21:53.874 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.875 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.450933342, 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.876 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.877 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:21:53.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:21:53.879 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:21:53.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.880 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.880 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.880 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:21:53.880 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:21:53.880 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.880 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.880 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.881 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.881 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.881 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:53.881 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.45056345099999, 2024-09-12T07:21:53.881 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:21:53.881 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.881 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.881 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.881 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.882 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:21:53.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.883 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.883 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:21:53.883 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:21:53.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.883 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.883 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.883 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:21:53.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:21:53.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.884 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.884 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.884 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:21:53.884 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:21:53.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:21:53.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:21:53.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.885 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.885 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:21:53.885 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:21:53.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.885 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.886 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.886 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:21:53.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:21:53.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.886 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.886 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.886 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:21:53.886 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:21:53.886 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.886 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.450373253, 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.887 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.888 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:21:53.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.888 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.888 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.888 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:53.888 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.888 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:21:53.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:21:53.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:21:53.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.889 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.889 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:21:53.889 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:21:53.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:21:53.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:21:53.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.890 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.890 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.890 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:21:53.890 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:21:53.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.891 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.45015317399999, 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.893 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:21:53.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:21:53.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:21:53.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.894 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.894 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.894 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:21:53.894 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.895 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:21:53.896 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.897 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.897 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.897 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.897 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:53.897 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.450009509, 2024-09-12T07:21:53.897 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:21:53.897 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.897 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.897 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.898 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.898 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.898 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.898 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:21:53.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.898 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.898 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.898 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:53.898 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:21:53.899 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.900 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.901 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.902 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.902 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:53.902 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.449783337, 2024-09-12T07:21:53.902 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:21:53.902 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.902 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.902 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.903 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.903 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.903 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.903 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:21:53.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.903 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.903 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.903 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:53.903 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.903 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:21:53.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.904 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:21:53.904 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:21:53.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.904 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:21:53.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.906 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.449582979, 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.907 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.908 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:21:53.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:21:53.910 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:21:53.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:21:53.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:21:53.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.911 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.911 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:21:53.911 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:21:53.911 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.911 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.912 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.912 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.912 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:53.912 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.449175096, 2024-09-12T07:21:53.912 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:21:53.912 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.914 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.914 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.914 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:21:53.914 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.914 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.914 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.914 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:21:53.914 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:21:53.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.915 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.915 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:21:53.915 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:21:53.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.915 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.915 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:21:53.915 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:21:53.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.916 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:21:53.916 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:21:53.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.916 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:21:53.916 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:21:53.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.916 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.916 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:21:53.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:21:53.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.917 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.917 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:21:53.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:21:53.917 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.917 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.917 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.918 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.918 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:53.918 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.448709214, 2024-09-12T07:21:53.918 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:21:53.918 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.918 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.918 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.918 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.918 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.918 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:21:53.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.919 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.919 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:53.919 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:21:53.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:21:53.920 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:21:53.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:21:53.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:21:53.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:21:53.921 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:21:53.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:21:53.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:21:53.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:21:53.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:21:53.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:21:53.922 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:21:53.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.923 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:21:53.923 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:21:53.923 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.923 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.923 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.923 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.447107004, 2024-09-12T07:21:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:21:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:21:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:21:53.925 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:21:53.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:21:53.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:21:53.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:21:53.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:21:53.927 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:21:53.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.928 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.928 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.928 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:21:53.928 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:21:53.928 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.928 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.928 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.928 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.928 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.929 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:53.929 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.446572969, 2024-09-12T07:21:53.929 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:21:53.929 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.929 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.929 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.929 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.929 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.929 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.929 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:21:53.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.930 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:53.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:21:53.930 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:21:53.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:21:53.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:21:53.931 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:21:53.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.932 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.932 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:21:53.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:21:53.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.932 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.932 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.932 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:21:53.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:21:53.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.932 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:21:53.933 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 114.19037489900001, 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:21:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:21:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:21:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:21:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:21:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:21:53.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.937 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:53.937 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:21:53.937 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:21:53.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.937 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:53.937 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:21:53.937 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:21:53.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.938 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:53.938 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:21:53.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:21:53.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:53.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:53.938 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:53.938 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:21:53.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:21:53.938 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.938 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.939 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.939 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:53.939 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:53.939 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:53.940 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:53.941 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:53.941+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:54.041 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:54.043+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:54.049 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:54.049 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:54.049 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:54.049 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:54.057 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:54.142 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:54.143+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:54.164 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:54.164 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:54.164 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:54.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:54.257 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:54.257+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:54.336 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:54.336 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.78818103099999, 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.337 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:21:54.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.341 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.788160494, 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:21:54.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:21:54.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.78814049100001, 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.346 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:21:54.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:21:54.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:21:54.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.350 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.350 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.350 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:21:54.350 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:21:54.350 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.351 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.351 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.351 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.351 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.351 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:54.351 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.788120542, 2024-09-12T07:21:54.351 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:21:54.351 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.351 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.351 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.351 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:54.352 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.353 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.354 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.355 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.78809967399999, 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.356 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:21:54.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:21:54.358 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.359 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.788077973, 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.361 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.361 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.361 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.361 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:21:54.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.361 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.361 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.361 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:54.361 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.362 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:21:54.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:21:54.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:21:54.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:21:54.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.363 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.363 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.363 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:21:54.363 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:21:54.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:21:54.364 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.788057533, 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.365 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.366 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.367 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.368 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.369 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.788029812, 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:54.370 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:21:54.371 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.372 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.373 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.788006877, 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.374 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.375 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:21:54.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:21:54.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:21:54.376 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:21:54.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.376 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.376 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.376 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:21:54.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:54.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.377 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:21:54.377 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:54.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.377 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.377 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.787986577, 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.379 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:21:54.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:21:54.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:21:54.381 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:21:54.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.381 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.381 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.382 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.383 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.787966301, 2024-09-12T07:21:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:21:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:21:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.385 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:54.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.385 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.385 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:54.385 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:21:54.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:21:54.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.787946142, 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.388 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:21:54.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.392 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.78780894400001, 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.393 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.396 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.78778629599999, 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:21:54.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.398 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.399 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.400 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.78132431, 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.401 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.402 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.403 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.404 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.405 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.781094673, 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.406 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.408 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:21:54.409 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.774170982, 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.410 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:21:54.412 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.414 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.773811399, 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.415 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.416 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:21:54.417 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.418 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 110.77373141299999, 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:54.419 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.420 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.421 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:21:54.422 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.423 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 32.844384161999997, 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0005192119999999, 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:21:54.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:54.425 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492882+0000", 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492888+0000", 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492904+0000", 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5628e-05 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.426 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492912+0000", 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4039999999999995e-06 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492962+0000", 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9425999999999997e-05 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492969+0000", 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1729999999999998e-06 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.427 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.493388+0000", 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0004188510000001 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.493399+0000", 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1571e-05 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:54.428 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:54.430 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:54.430+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:54.430 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:54.530 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:54.531+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:54.539 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:54.539 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:54.539 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:54.539 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:54.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:54.631 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:54.633+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:54.654 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:54.655 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:54.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:54.732 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:54.733+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:54.812 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:54.813 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:54.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:54.813 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:54.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.813 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:148945 3.10 3:0cac0a45:::rbd_data.1160161ce0c8.000000000000009d:head [delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-12T07:21:54.813 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:54.813 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.379584214000005, 2024-09-12T07:21:54.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89878390500000005, 2024-09-12T07:21:54.813 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.813 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 148945 2024-09-12T07:21:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433803+0000", 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.164e-06 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433807+0000", 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5000000000000001e-06 2024-09-12T07:21:54.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433816+0000", 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1859999999999998e-06 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433822+0000", 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5300000000000004e-06 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433886+0000", 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4510000000000004e-05 2024-09-12T07:21:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433896+0000", 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0012e-05 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329359+0000", 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89546257399999996 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329585+0000", 2024-09-12T07:21:54.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022641800000000001 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329712+0000", 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126339 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.330704+0000", 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099219900000000003 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:54.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332513+0000", 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001808689 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332544+0000", 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1186999999999998e-05 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332585+0000", 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1597000000000002e-05 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.819 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170165 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1101824~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.252023293999997, 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073287456000000001, 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170165 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561363+0000", 2024-09-12T07:21:54.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.96e-07 2024-09-12T07:21:54.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561365+0000", 2024-09-12T07:21:54.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.181e-06 2024-09-12T07:21:54.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561371+0000", 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561377+0000", 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.536e-06 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634513+0000", 2024-09-12T07:21:54.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073136229999999997 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634548+0000", 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5206999999999997e-05 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634650+0000", 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101678 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170166 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.251989954000003, 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073346145000000001, 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170166 2024-09-12T07:21:54.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561399+0000", 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4049999999999998e-06 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561405+0000", 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5370000000000003e-06 2024-09-12T07:21:54.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561410+0000", 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2889999999999997e-06 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634666+0000", 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073255526000000001 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634701+0000", 2024-09-12T07:21:54.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5197999999999997e-05 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634742+0000", 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0757999999999998e-05 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170167 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1183744~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.251967463, 2024-09-12T07:21:54.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073410903, 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170167 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561419+0000", 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3000000000000002e-07 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561421+0000", 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1059999999999998e-06 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561427+0000", 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0970000000000002e-06 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561432+0000", 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1120000000000004e-06 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634756+0000", 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073324571000000005 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634790+0000", 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3081999999999999e-05 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634829+0000", 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9604999999999998e-05 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170168 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1224704~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.833 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.251947213000001, 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073476793999999998, 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170168 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561439+0000", 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561441+0000", 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8920000000000001e-06 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561447+0000", 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2360000000000001e-06 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561452+0000", 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9810000000000003e-06 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634843+0000", 2024-09-12T07:21:54.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073391434000000005 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634876+0000", 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2869e-05 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634915+0000", 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9001999999999997e-05 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170169 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1265664~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.251927334999998, 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073551819000000004, 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170169 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561459+0000", 2024-09-12T07:21:54.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.77e-07 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561460+0000", 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.733e-06 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561468+0000", 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561474+0000", 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4829999999999999e-06 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634930+0000", 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073456244000000004 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634969+0000", 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9687000000000001e-05 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635010+0000", 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0685000000000001e-05 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170170 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1306624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.251904940000003, 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073616897000000001, 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170170 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:54.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2e-07 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561483+0000", 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0619999999999999e-06 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561491+0000", 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7880000000000007e-06 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561497+0000", 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5480000000000001e-06 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635025+0000", 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073528643000000005 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635058+0000", 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3077999999999997e-05 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635098+0000", 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170171 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1347584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.846 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.251882174999999, 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073682407000000005, 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170171 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561506+0000", 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.441e-06 2024-09-12T07:21:54.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561535+0000", 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8827999999999999e-05 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561542+0000", 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7100000000000001e-06 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635112+0000", 2024-09-12T07:21:54.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073569967999999999 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635146+0000", 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4413999999999999e-05 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635186+0000", 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9625000000000001e-05 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170172 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.251834821000003, 2024-09-12T07:21:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073720519999999998, 2024-09-12T07:21:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170172 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1500000000000005e-07 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561554+0000", 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561563+0000", 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7760000000000003e-06 2024-09-12T07:21:54.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561568+0000", 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.305e-06 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635200+0000", 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073632141999999998 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635233+0000", 2024-09-12T07:21:54.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.239e-05 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635271+0000", 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8688e-05 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170173 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1429504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:54.855 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.251810233, 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07378113, 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170173 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:54.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561576+0000", 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999998e-07 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561578+0000", 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0389999999999999e-06 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561587+0000", 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0999999999999993e-06 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561594+0000", 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.844e-06 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635286+0000", 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073691799000000002 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635318+0000", 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2240999999999998e-05 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635357+0000", 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8726000000000002e-05 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170174 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1470464~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.251782042000002, 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073841247999999998, 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170174 2024-09-12T07:21:54.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.861 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:54.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:54.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:54.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:21:54.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561607+0000", 2024-09-12T07:21:54.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:54.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561613+0000", 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3339999999999998e-06 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561618+0000", 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.203e-06 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635371+0000", 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073752811000000001 2024-09-12T07:21:54.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635404+0000", 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3250000000000002e-05 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635445+0000", 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0741000000000002e-05 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.865 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170175 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1511424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.865 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:54.865 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.251760421, 2024-09-12T07:21:54.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073939900000000003, 2024-09-12T07:21:54.865 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.865 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.865 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.865 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.865 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.865 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170175 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561626+0000", 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561632+0000", 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8290000000000002e-06 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561642+0000", 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1209999999999996e-06 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561647+0000", 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5119999999999999e-06 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635459+0000", 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073811962999999994 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635521+0000", 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2323000000000006e-05 2024-09-12T07:21:54.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635565+0000", 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3763000000000001e-05 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170176 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1552384~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.251723370999997, 2024-09-12T07:21:54.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.074029015000000004, 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170176 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561663+0000", 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561665+0000", 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:21:54.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561672+0000", 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4930000000000003e-06 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561677+0000", 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3159999999999996e-06 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635587+0000", 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073909937999999994 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635629+0000", 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2555999999999997e-05 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635691+0000", 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1946000000000006e-05 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.873 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170183 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1757184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.249769471999997, 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073321546000000001, 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170183 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563617+0000", 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999999e-07 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563619+0000", 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.153e-06 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563625+0000", 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3160000000000002e-06 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563632+0000", 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9820000000000002e-06 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636752+0000", 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073120428000000001 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636838+0000", 2024-09-12T07:21:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5177000000000001e-05 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636938+0000", 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010008 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170185 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1798144~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.249451974000003, 2024-09-12T07:21:54.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073255606000000001, 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170185 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5600000000000005e-07 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563989+0000", 2024-09-12T07:21:54.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4993e-05 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563998+0000", 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8710000000000003e-06 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564005+0000", 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8449999999999997e-06 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636991+0000", 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072986350000000005 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637076+0000", 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4248999999999999e-05 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637189+0000", 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011364199999999999 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170186 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1839104~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.249368416000003, 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073312293000000001, 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170186 2024-09-12T07:21:54.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564018+0000", 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.99e-07 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564021+0000", 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7410000000000001e-06 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564027+0000", 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5119999999999997e-06 2024-09-12T07:21:54.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564033+0000", 2024-09-12T07:21:54.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4970000000000001e-06 2024-09-12T07:21:54.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637230+0000", 2024-09-12T07:21:54.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073197613999999994 2024-09-12T07:21:54.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637271+0000", 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1001000000000003e-05 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637330+0000", 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8328999999999997e-05 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170189 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1921024~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.249033286, 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073393609999999998, 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170189 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.888 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564353+0000", 2024-09-12T07:21:54.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9500000000000003e-07 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564355+0000", 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1409999999999999e-06 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564362+0000", 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6279999999999999e-06 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564367+0000", 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5520000000000004e-06 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637563+0000", 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073196285 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637641+0000", 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7051000000000005e-05 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637746+0000", 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010545800000000001 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170191 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1961984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.248750324, 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073246585000000003, 2024-09-12T07:21:54.892 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170191 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564636+0000", 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6400000000000002e-07 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564639+0000", 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.399e-06 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564646+0000", 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0999999999999998e-06 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564653+0000", 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9659999999999999e-06 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637785+0000", 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073131357999999994 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637826+0000", 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1742000000000001e-05 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637882+0000", 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5455999999999999e-05 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.896 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170192 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 2002944~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.248720310000003, 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073448428999999996, 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170192 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564666+0000", 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564668+0000", 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564675+0000", 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2230000000000004e-06 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564680+0000", 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3210000000000004e-06 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637923+0000", 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073243483999999998 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638011+0000", 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7149000000000002e-05 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638114+0000", 2024-09-12T07:21:54.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103278 2024-09-12T07:21:54.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.901 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.901 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170198 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 2084864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:54.901 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 45.248157440999996, 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073259037999999999, 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170198 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565229+0000", 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0200000000000002e-07 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565231+0000", 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2330000000000001e-06 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565238+0000", 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.596e-06 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565243+0000", 2024-09-12T07:21:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.31e-06 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638324+0000", 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073080720000000002 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638391+0000", 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7037000000000002e-05 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638487+0000", 2024-09-12T07:21:54.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6639999999999996e-05 2024-09-12T07:21:54.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.906 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:54.906 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:54.906 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:54.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:54.908 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:54.909+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:54.946 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 6, 7, 3] out_osds: [1, 5, 4] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-12T07:21:54.946 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:21:54.947 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-12T07:21:54.947 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-12T07:21:55.008 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:55.008+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:55.058 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:55.058 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:55.058 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:55.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:55.111 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:55.113+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:55.183 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:55.183 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:55.183 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:55.183 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:55.184 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:55.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:55.210 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:55.210+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:55.263 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T07:21:55.284 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-12T07:21:55.286+0000 7f0757424640 -1 mon.a@0(leader).osd e152 definitely_dead 0 2024-09-12T07:21:55.311 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:55.312+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:55.347 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 111.965056804, 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:21:55.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:21:55.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:21:55.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 111.90197854, 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:21:55.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:21:55.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:21:55.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:21:55.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:21:55.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:21:55.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:21:55.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:21:55.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 111.89827345499999, 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.360 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:21:55.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:21:55.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:55.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:21:55.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:21:55.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.366 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.366 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:21:55.366 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:55.366 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 107.89992717600001, 2024-09-12T07:21:55.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:21:55.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116552 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5267456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:55.371 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.880597824000006, 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034412801999999999, 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116552 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:55.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0400000000000002e-07 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467850+0000", 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6529999999999998e-06 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467856+0000", 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467861+0000", 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502080+0000", 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034218947999999999 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502159+0000", 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9121000000000004e-05 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502260+0000", 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100846 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116553 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5472256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.880577125000002, 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034609677999999998, 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116553 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467868+0000", 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6800000000000001e-07 2024-09-12T07:21:55.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467870+0000", 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.773e-06 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467876+0000", 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0079999999999999e-06 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467880+0000", 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502304+0000", 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423924000000002 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502379+0000", 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5010000000000002e-05 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502477+0000", 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7942999999999994e-05 2024-09-12T07:21:55.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116554 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5308416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.880557585000005, 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034779583000000003, 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116554 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467889+0000", 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7430000000000001e-06 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467895+0000", 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8159999999999999e-06 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467900+0000", 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502507+0000", 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034607252999999998 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502580+0000", 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3337999999999995e-05 2024-09-12T07:21:55.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502667+0000", 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6468e-05 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116555 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5349376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.880538611000006, 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034965729000000001, 2024-09-12T07:21:55.384 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116555 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1899999999999998e-07 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467908+0000", 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5740000000000001e-06 2024-09-12T07:21:55.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467914+0000", 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9939999999999997e-06 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467918+0000", 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.622e-06 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502696+0000", 2024-09-12T07:21:55.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034777798999999998 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502780+0000", 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3551e-05 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502872+0000", 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.187e-05 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116557 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3506176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.879444196999998, 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034335510999999999, 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116557 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.389 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469001+0000", 2024-09-12T07:21:55.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469003+0000", 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469012+0000", 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8030000000000003e-06 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469019+0000", 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6959999999999999e-06 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503153+0000", 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034134096000000003 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503236+0000", 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2886000000000006e-05 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503336+0000", 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9919000000000003e-05 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116558 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3469312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.879413916000004, 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034519743999999998, 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116558 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469034+0000", 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469042+0000", 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469047+0000", 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503370+0000", 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034323201999999997 2024-09-12T07:21:55.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503454+0000", 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4109000000000004e-05 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503550+0000", 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6009999999999997e-05 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116559 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3551232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.879387084000001, 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034689659999999997, 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116559 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469058+0000", 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469061+0000", 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.497e-06 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469069+0000", 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2579999999999995e-06 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469074+0000", 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1279999999999999e-06 2024-09-12T07:21:55.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503582+0000", 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034508351 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503656+0000", 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3234999999999999e-05 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503747+0000", 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1638999999999995e-05 2024-09-12T07:21:55.401 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116560 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.879360356000007, 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034902912000000001, 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116560 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469085+0000", 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4700000000000001e-07 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469087+0000", 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4760000000000001e-06 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469095+0000", 2024-09-12T07:21:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2339999999999994e-06 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469100+0000", 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503796+0000", 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034695411000000002 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503868+0000", 2024-09-12T07:21:55.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2032000000000004e-05 2024-09-12T07:21:55.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503987+0000", 2024-09-12T07:21:55.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119369 2024-09-12T07:21:55.406 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.406 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116561 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3592192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.879336957000007, 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035102170000000002, 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116561 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469110+0000", 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1629999999999999e-06 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469116+0000", 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.251e-06 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469121+0000", 2024-09-12T07:21:55.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8210000000000001e-06 2024-09-12T07:21:55.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504017+0000", 2024-09-12T07:21:55.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034895772999999998 2024-09-12T07:21:55.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504108+0000", 2024-09-12T07:21:55.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1027999999999997e-05 2024-09-12T07:21:55.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504210+0000", 2024-09-12T07:21:55.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101729 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116562 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.879316113000002, 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035285642999999998, 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116562 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469129+0000", 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469131+0000", 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1550000000000001e-06 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469138+0000", 2024-09-12T07:21:55.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5520000000000001e-06 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469142+0000", 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9099999999999996e-06 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504239+0000", 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.03509669 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504311+0000", 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1650999999999996e-05 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504414+0000", 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103286 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116563 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.417 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.879295479999996, 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035471691, 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116563 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469153+0000", 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.05e-06 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469158+0000", 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.874e-06 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469163+0000", 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8069999999999999e-06 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504444+0000", 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035280793999999997 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504523+0000", 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9036999999999996e-05 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504621+0000", 2024-09-12T07:21:55.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7743000000000003e-05 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116564 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5435392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.879274369000001, 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035734657000000003, 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116564 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:55.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469171+0000", 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469172+0000", 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.832e-06 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469179+0000", 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1449999999999996e-06 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469184+0000", 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504721+0000", 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035536890000000002 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504809+0000", 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7229000000000001e-05 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504905+0000", 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6337999999999998e-05 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116565 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5476352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:55.426 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.879253703000003, 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035933158, 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116565 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469193+0000", 2024-09-12T07:21:55.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.973e-06 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469201+0000", 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3730000000000004e-06 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469206+0000", 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1660000000000002e-06 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504937+0000", 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035731205000000002 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505035+0000", 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7759999999999999e-05 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505124+0000", 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9258000000000005e-05 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.430 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116566 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5271552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.879231117000003, 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036116848999999999, 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116566 2024-09-12T07:21:55.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469214+0000", 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:55.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469216+0000", 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8759999999999999e-06 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469222+0000", 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.173e-06 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469227+0000", 2024-09-12T07:21:55.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3070000000000002e-06 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505154+0000", 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035926362000000003 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505239+0000", 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5261999999999998e-05 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505330+0000", 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1436999999999999e-05 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116567 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5312512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.879210588000007, 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036305875000000001, 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116567 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.58e-07 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469242+0000", 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469249+0000", 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5549999999999999e-06 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469254+0000", 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9899999999999997e-06 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505361+0000", 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036106923999999999 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505449+0000", 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8531999999999999e-05 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505540+0000", 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0805999999999998e-05 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116568 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5353472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:55.439 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.879183914999999, 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036561383000000003, 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116568 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469263+0000", 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7549999999999999e-06 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469269+0000", 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7219999999999996e-06 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469273+0000", 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505631+0000", 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036357997000000003 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505711+0000", 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9331000000000003e-05 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505822+0000", 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000111208 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:55.444 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:55.444 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:55.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:55.445 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:55.446+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:55.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:21:55.546 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:55.547+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:55.597 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:55.597 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:55.597 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:55.597 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:55.606 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:55.613 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:55.613 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:55.631 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-12T07:21:55.646 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:55.648+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:55.648 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:55.665 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:55.683 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:55.683 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:55.701 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:55.715 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:55.715 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:55.718 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:55.718 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:55.722 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T07:21:55.722 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T07:21:55.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:55.766 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:55.767+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:55.848 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:55.849+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:55.872 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:55.872 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:55.880 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:55.950 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:55.950+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:55.994 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:55.995 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:56.003 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:56.050 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:56.051+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:56.150 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:56.150 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:56.151 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:56.151+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:56.159 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:56.251 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:56.252+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:56.254 INFO:teuthology.orchestra.run.smithi046.stderr:marked in osd.4. 2024-09-12T07:21:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is set 2024-09-12T07:21:56.276 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-12T07:21:56.311 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:56.311 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:56.320 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:56.353 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:56.354+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:56.453 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:56.455+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:56.468 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:56.468 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:56.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:56.554 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:56.554+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:56.582 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:56.583 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:56.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:21:56.654 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:56.656+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:56.738 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:56.738 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:56.747 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:21:56.755 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:56.757+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:56.854 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:56.854 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:56.856 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:56.857+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:56.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:21:56.957 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:56.958+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:57.010 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:57.010 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:57.018 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:21:57.058 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:57.060+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:57.159 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:57.161+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:57.165 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:57.165 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:57.175 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:21:57.260 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:57.261+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:57.286 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:57.286 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:57.295 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:21:57.361 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:57.363+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:57.402 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:57.402 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:57.410 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:21:57.462 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:57.463+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:57.517 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:57.518 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:57.527 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:21:57.571 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:21:57.573+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:21:57.648 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:57.648 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:57.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:21:57.664 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:57.664+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:57.723 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:21:57.764 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:57.766+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:57.809 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:21:57.809 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:21:57.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:21:57.866 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:57.867+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:57.868 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:21:57.868 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:21:57.885 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:21:57.886 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:21:57.886 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-12T07:21:57.903 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:21:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:21:57.920 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:21:57.920 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:21:57.928 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:57.928 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:148945 3.10 3:0cac0a45:::rbd_data.1160161ce0c8.000000000000009d:head [delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.495566835000005, 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89878390500000005, 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.929 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 148945 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433801+0000", 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433803+0000", 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.164e-06 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433807+0000", 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5000000000000001e-06 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433816+0000", 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1859999999999998e-06 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433822+0000", 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5300000000000004e-06 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433886+0000", 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4510000000000004e-05 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:36.433896+0000", 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0012e-05 2024-09-12T07:21:57.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329359+0000", 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.89546257399999996 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329585+0000", 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022641800000000001 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.329712+0000", 2024-09-12T07:21:57.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126339 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.330704+0000", 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099219900000000003 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332513+0000", 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001808689 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332544+0000", 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1186999999999998e-05 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:37.332585+0000", 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1597000000000002e-05 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170165 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1101824~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:57.935 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.368005914999998, 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073287456000000001, 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170165 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561362+0000", 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561363+0000", 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.96e-07 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561365+0000", 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.181e-06 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561371+0000", 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2280000000000003e-06 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561377+0000", 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.536e-06 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634513+0000", 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073136229999999997 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634548+0000", 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5206999999999997e-05 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634650+0000", 2024-09-12T07:21:57.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101678 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170166 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.367972575000003, 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073346145000000001, 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170166 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561396+0000", 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561399+0000", 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4049999999999998e-06 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561405+0000", 2024-09-12T07:21:57.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5370000000000003e-06 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561410+0000", 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2889999999999997e-06 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634666+0000", 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073255526000000001 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634701+0000", 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5197999999999997e-05 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634742+0000", 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0757999999999998e-05 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170167 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1183744~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.944 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.367950084, 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073410903, 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170167 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:57.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561418+0000", 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561419+0000", 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3000000000000002e-07 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561421+0000", 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1059999999999998e-06 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561427+0000", 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0970000000000002e-06 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561432+0000", 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1120000000000004e-06 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634756+0000", 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073324571000000005 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634790+0000", 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3081999999999999e-05 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634829+0000", 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9604999999999998e-05 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.948 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170168 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1224704~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.367929834000002, 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073476793999999998, 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.949 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170168 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561438+0000", 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561439+0000", 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561441+0000", 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8920000000000001e-06 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561447+0000", 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2360000000000001e-06 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561452+0000", 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9810000000000003e-06 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634843+0000", 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073391434000000005 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634876+0000", 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2869e-05 2024-09-12T07:21:57.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634915+0000", 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9001999999999997e-05 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170169 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1265664~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.367909955999998, 2024-09-12T07:21:57.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073551819000000004, 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170169 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561458+0000", 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561459+0000", 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.77e-07 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561460+0000", 2024-09-12T07:21:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.733e-06 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561468+0000", 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561474+0000", 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4829999999999999e-06 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634930+0000", 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073456244000000004 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.634969+0000", 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9687000000000001e-05 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635010+0000", 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0685000000000001e-05 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.957 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170170 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1306624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.367887561000003, 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073616897000000001, 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.958 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170170 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561481+0000", 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2e-07 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561483+0000", 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0619999999999999e-06 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561491+0000", 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7880000000000007e-06 2024-09-12T07:21:57.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561497+0000", 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5480000000000001e-06 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635025+0000", 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073528643000000005 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635058+0000", 2024-09-12T07:21:57.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3077999999999997e-05 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635098+0000", 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9357999999999998e-05 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170171 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1347584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:57.962 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.367864795999999, 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073682407000000005, 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170171 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:57.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561504+0000", 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561506+0000", 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.441e-06 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561535+0000", 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8827999999999999e-05 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561542+0000", 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7100000000000001e-06 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635112+0000", 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073569967999999999 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635146+0000", 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4413999999999999e-05 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635186+0000", 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9625000000000001e-05 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170172 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.367817442000003, 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073720519999999998, 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170172 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561551+0000", 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1500000000000005e-07 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561554+0000", 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7039999999999999e-06 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561563+0000", 2024-09-12T07:21:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7760000000000003e-06 2024-09-12T07:21:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561568+0000", 2024-09-12T07:21:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.305e-06 2024-09-12T07:21:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635200+0000", 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073632141999999998 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635233+0000", 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.239e-05 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635271+0000", 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8688e-05 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170173 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1429504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.367792854000001, 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.07378113, 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170173 2024-09-12T07:21:57.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561575+0000", 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561576+0000", 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999998e-07 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561578+0000", 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0389999999999999e-06 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561587+0000", 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0999999999999993e-06 2024-09-12T07:21:57.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561594+0000", 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.844e-06 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635286+0000", 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073691799000000002 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635318+0000", 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2240999999999998e-05 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635357+0000", 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8726000000000002e-05 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170174 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1470464~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.976 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.367764663000003, 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073841247999999998, 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170174 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561604+0000", 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2100000000000002e-07 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561607+0000", 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561613+0000", 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3339999999999998e-06 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561618+0000", 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.203e-06 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635371+0000", 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073752811000000001 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635404+0000", 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3250000000000002e-05 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635445+0000", 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0741000000000002e-05 2024-09-12T07:21:57.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170175 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1511424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.367743042000001, 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073939900000000003, 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.981 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170175 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561625+0000", 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561626+0000", 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561632+0000", 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8290000000000002e-06 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561642+0000", 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1209999999999996e-06 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561647+0000", 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5119999999999999e-06 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635459+0000", 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073811962999999994 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635521+0000", 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2323000000000006e-05 2024-09-12T07:21:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635565+0000", 2024-09-12T07:21:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3763000000000001e-05 2024-09-12T07:21:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170176 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1552384~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.367705991999998, 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.074029015000000004, 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170176 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561662+0000", 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561663+0000", 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561665+0000", 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561672+0000", 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4930000000000003e-06 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.561677+0000", 2024-09-12T07:21:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3159999999999996e-06 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635587+0000", 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073909937999999994 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635629+0000", 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2555999999999997e-05 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.635691+0000", 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1946000000000006e-05 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170183 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1757184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.365752092999998, 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073321546000000001, 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170183 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563616+0000", 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563617+0000", 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999999e-07 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563619+0000", 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.153e-06 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563625+0000", 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3160000000000002e-06 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563632+0000", 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9820000000000002e-06 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636752+0000", 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073120428000000001 2024-09-12T07:21:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636838+0000", 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5177000000000001e-05 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636938+0000", 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010008 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170185 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1798144~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.365434594999996, 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073255606000000001, 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170185 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563934+0000", 2024-09-12T07:21:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5600000000000005e-07 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563989+0000", 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4993e-05 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.563998+0000", 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8710000000000003e-06 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564005+0000", 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8449999999999997e-06 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.636991+0000", 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.072986350000000005 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637076+0000", 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4248999999999999e-05 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637189+0000", 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011364199999999999 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170186 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1839104~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.365351037000003, 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073312293000000001, 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170186 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564017+0000", 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564018+0000", 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.99e-07 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564021+0000", 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7410000000000001e-06 2024-09-12T07:21:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564027+0000", 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5119999999999997e-06 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564033+0000", 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4970000000000001e-06 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637230+0000", 2024-09-12T07:21:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073197613999999994 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637271+0000", 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1001000000000003e-05 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637330+0000", 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8328999999999997e-05 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170189 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1921024~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.365015907, 2024-09-12T07:21:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073393609999999998, 2024-09-12T07:21:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170189 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564352+0000", 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564353+0000", 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9500000000000003e-07 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564355+0000", 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1409999999999999e-06 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564362+0000", 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6279999999999999e-06 2024-09-12T07:21:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564367+0000", 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5520000000000004e-06 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637563+0000", 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073196285 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637641+0000", 2024-09-12T07:21:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7051000000000005e-05 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637746+0000", 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010545800000000001 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170191 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 1961984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.364732945, 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073246585000000003, 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170191 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564635+0000", 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564636+0000", 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6400000000000002e-07 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564639+0000", 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.399e-06 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564646+0000", 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0999999999999998e-06 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564653+0000", 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9659999999999999e-06 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637785+0000", 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073131357999999994 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637826+0000", 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1742000000000001e-05 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637882+0000", 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5455999999999999e-05 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170192 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 2002944~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.364702930999997, 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073448428999999996, 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170192 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564665+0000", 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564666+0000", 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2899999999999999e-07 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564668+0000", 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564675+0000", 2024-09-12T07:21:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2230000000000004e-06 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.564680+0000", 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3210000000000004e-06 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.637923+0000", 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073243483999999998 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638011+0000", 2024-09-12T07:21:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7149000000000002e-05 2024-09-12T07:21:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638114+0000", 2024-09-12T07:21:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103278 2024-09-12T07:21:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:170198 3.12 3:4ec007fc:::rbd_data.1160559f6e27.0000000000000002:2f [read 2084864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.364140061999997, 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073259037999999999, 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170198 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565228+0000", 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565229+0000", 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0200000000000002e-07 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565231+0000", 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2330000000000001e-06 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565238+0000", 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.596e-06 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.565243+0000", 2024-09-12T07:21:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.31e-06 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638324+0000", 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.073080720000000002 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638391+0000", 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7037000000000002e-05 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.638487+0000", 2024-09-12T07:21:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6639999999999996e-05 2024-09-12T07:21:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.022 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:58.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:21:58.024 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:21:58.024 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:21:58.025 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:21:58.025 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:21:58.025 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:21:58.026 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T07:21:58.026 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:21:58.026 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T07:21:58.027 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:58.024+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:58.124 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:58.125+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:58.178 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:58.178 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:58.178 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:58.178 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:58.183 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:21:58.225 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:58.226+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:58.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:21:58.296 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:58.296 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:58.296 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:58.305 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:21:58.326 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:58.326+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:58.426 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:58.428+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:58.476 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:66595 3.8 3:13e302ea:::rbd_data.1160f022dd61.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 115.094113487, 2024-09-12T07:21:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.038084777, 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 66595 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383388+0000", 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.383390+0000", 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.103e-06 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389381+0000", 2024-09-12T07:21:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059908000000000001 2024-09-12T07:21:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389406+0000", 2024-09-12T07:21:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5040000000000001e-05 2024-09-12T07:21:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389417+0000", 2024-09-12T07:21:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1053000000000001e-05 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389493+0000", 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6046999999999999e-05 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389609+0000", 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115828 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-12T07:21:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.389994+0000", 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038583399999999999 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.412668+0000", 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022673427999999999 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421345+0000", 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086767259999999992 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421423+0000", 2024-09-12T07:21:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8155e-05 2024-09-12T07:21:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.421473+0000", 2024-09-12T07:21:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9762999999999998e-05 2024-09-12T07:21:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67519 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 115.031035223, 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.045203587000000003, 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67519 2024-09-12T07:21:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446466+0000", 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.446468+0000", 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2249999999999999e-06 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450099+0000", 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003631053 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450123+0000", 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4068000000000001e-05 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450134+0000", 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1001e-05 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450299+0000", 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016474600000000001 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450396+0000", 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7108999999999995e-05 2024-09-12T07:21:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.451337+0000", 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094042400000000003 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.477118+0000", 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.025781700000000001 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491554+0000", 2024-09-12T07:21:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014435536000000001 2024-09-12T07:21:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491620+0000", 2024-09-12T07:21:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6494000000000004e-05 2024-09-12T07:21:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491670+0000", 2024-09-12T07:21:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230999999999997e-05 2024-09-12T07:21:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:67520 3.0 3:0598eb6c:::rbd_data.1160f022dd61.0000000000000009:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 115.027330138, 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041826435000000002, 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 67520 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450171+0000", 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450173+0000", 2024-09-12T07:21:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8339999999999999e-06 2024-09-12T07:21:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450178+0000", 2024-09-12T07:21:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9819999999999999e-06 2024-09-12T07:21:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450190+0000", 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.450196+0000", 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5089999999999999e-06 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466190+0000", 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015993974000000001 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466356+0000", 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000165954 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.466636+0000", 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027985599999999998 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.478523+0000", 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011886724 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491894+0000", 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013371015 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491952+0000", 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7935e-05 2024-09-12T07:21:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:03.491998+0000", 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5697e-05 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:80197 3.19 3:9be84e8c:::rbd_object_map.11604a4c57c2.000000000000000a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 111.02898385899999, 2024-09-12T07:21:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74501472499999999, 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 80197 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448517+0000", 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448518+0000", 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.065e-06 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448523+0000", 2024-09-12T07:21:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0180000000000001e-06 2024-09-12T07:21:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448535+0000", 2024-09-12T07:21:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1598e-05 2024-09-12T07:21:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448541+0000", 2024-09-12T07:21:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7799999999999997e-06 2024-09-12T07:21:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448598+0000", 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6780000000000002e-05 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:07.448604+0000", 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2389999999999999e-06 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193017+0000", 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.74441283000000003 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193221+0000", 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000204358 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:08.193532+0000", 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000311057 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116552 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5267456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.009654506999993, 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034412801999999999, 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116552 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467847+0000", 2024-09-12T07:21:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0400000000000002e-07 2024-09-12T07:21:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467850+0000", 2024-09-12T07:21:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6529999999999998e-06 2024-09-12T07:21:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467856+0000", 2024-09-12T07:21:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:21:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467861+0000", 2024-09-12T07:21:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7060000000000003e-06 2024-09-12T07:21:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502080+0000", 2024-09-12T07:21:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034218947999999999 2024-09-12T07:21:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502159+0000", 2024-09-12T07:21:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9121000000000004e-05 2024-09-12T07:21:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502260+0000", 2024-09-12T07:21:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100846 2024-09-12T07:21:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116553 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5472256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.009633808000004, 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034609677999999998, 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116553 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467867+0000", 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467868+0000", 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6800000000000001e-07 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467870+0000", 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.773e-06 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467876+0000", 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0079999999999999e-06 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467880+0000", 2024-09-12T07:21:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502304+0000", 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423924000000002 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502379+0000", 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5010000000000002e-05 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502477+0000", 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7942999999999994e-05 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116554 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5308416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.009614268000007, 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034779583000000003, 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116554 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467887+0000", 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999998e-07 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467889+0000", 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7430000000000001e-06 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467895+0000", 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8159999999999999e-06 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467900+0000", 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6149999999999999e-06 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502507+0000", 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034607252999999998 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502580+0000", 2024-09-12T07:21:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3337999999999995e-05 2024-09-12T07:21:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502667+0000", 2024-09-12T07:21:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6468e-05 2024-09-12T07:21:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116555 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5349376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.009595293999993, 2024-09-12T07:21:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034965729000000001, 2024-09-12T07:21:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116555 2024-09-12T07:21:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467906+0000", 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1899999999999998e-07 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467908+0000", 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5740000000000001e-06 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467914+0000", 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9939999999999997e-06 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.467918+0000", 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.622e-06 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502696+0000", 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034777798999999998 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502780+0000", 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3551e-05 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.502872+0000", 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.187e-05 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116557 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3506176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.00850088, 2024-09-12T07:21:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034335510999999999, 2024-09-12T07:21:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116557 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469000+0000", 2024-09-12T07:21:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469001+0000", 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6599999999999996e-07 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469003+0000", 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5450000000000002e-06 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469012+0000", 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8030000000000003e-06 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469019+0000", 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6959999999999999e-06 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503153+0000", 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034134096000000003 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503236+0000", 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2886000000000006e-05 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503336+0000", 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9919000000000003e-05 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116558 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3469312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.008470599000006, 2024-09-12T07:21:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034519743999999998, 2024-09-12T07:21:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116558 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469031+0000", 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469034+0000", 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.48e-06 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469042+0000", 2024-09-12T07:21:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469047+0000", 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0139999999999998e-06 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503370+0000", 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034323201999999997 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503454+0000", 2024-09-12T07:21:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4109000000000004e-05 2024-09-12T07:21:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503550+0000", 2024-09-12T07:21:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6009999999999997e-05 2024-09-12T07:21:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116559 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3551232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.008443767000003, 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034689659999999997, 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116559 2024-09-12T07:21:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469057+0000", 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469058+0000", 2024-09-12T07:21:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:21:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469061+0000", 2024-09-12T07:21:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.497e-06 2024-09-12T07:21:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469069+0000", 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2579999999999995e-06 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469074+0000", 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1279999999999999e-06 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503582+0000", 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034508351 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503656+0000", 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3234999999999999e-05 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503747+0000", 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1638999999999995e-05 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116560 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.008417038999994, 2024-09-12T07:21:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034902912000000001, 2024-09-12T07:21:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116560 2024-09-12T07:21:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469084+0000", 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469085+0000", 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4700000000000001e-07 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469087+0000", 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4760000000000001e-06 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469095+0000", 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2339999999999994e-06 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469100+0000", 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503796+0000", 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034695411000000002 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503868+0000", 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2032000000000004e-05 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.503987+0000", 2024-09-12T07:21:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000119369 2024-09-12T07:21:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116561 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3592192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.008393639999994, 2024-09-12T07:21:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035102170000000002, 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116561 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469108+0000", 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0499999999999999e-07 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469110+0000", 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1629999999999999e-06 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469116+0000", 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.251e-06 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469121+0000", 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8210000000000001e-06 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504017+0000", 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034895772999999998 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504108+0000", 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1027999999999997e-05 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504210+0000", 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101729 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116562 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.008372796000003, 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035285642999999998, 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116562 2024-09-12T07:21:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469128+0000", 2024-09-12T07:21:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469129+0000", 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9900000000000001e-07 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469131+0000", 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1550000000000001e-06 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469138+0000", 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5520000000000001e-06 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469142+0000", 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9099999999999996e-06 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504239+0000", 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.03509669 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504311+0000", 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1650999999999996e-05 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504414+0000", 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103286 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116563 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.008352162999998, 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035471691, 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116563 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469149+0000", 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8599999999999999e-07 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469153+0000", 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.05e-06 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469158+0000", 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.874e-06 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469163+0000", 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8069999999999999e-06 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504444+0000", 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035280793999999997 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504523+0000", 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9036999999999996e-05 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504621+0000", 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7743000000000003e-05 2024-09-12T07:21:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116564 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5435392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.008331052000003, 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035734657000000003, 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116564 2024-09-12T07:21:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469170+0000", 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469171+0000", 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-07 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469172+0000", 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.832e-06 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469179+0000", 2024-09-12T07:21:58.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1449999999999996e-06 2024-09-12T07:21:58.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469184+0000", 2024-09-12T07:21:58.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8340000000000004e-06 2024-09-12T07:21:58.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504721+0000", 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035536890000000002 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504809+0000", 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7229000000000001e-05 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504905+0000", 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6337999999999998e-05 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.559 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116565 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5476352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.008310386000005, 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035933158, 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116565 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469191+0000", 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2300000000000002e-07 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469193+0000", 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.973e-06 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469201+0000", 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3730000000000004e-06 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469206+0000", 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1660000000000002e-06 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.504937+0000", 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035731205000000002 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505035+0000", 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7759999999999999e-05 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505124+0000", 2024-09-12T07:21:58.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9258000000000005e-05 2024-09-12T07:21:58.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.564 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.564 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116566 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5271552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.564 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.008287800000005, 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036116848999999999, 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116566 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:58.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469213+0000", 2024-09-12T07:21:58.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469214+0000", 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.32e-07 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469216+0000", 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8759999999999999e-06 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469222+0000", 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.173e-06 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469227+0000", 2024-09-12T07:21:58.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3070000000000002e-06 2024-09-12T07:21:58.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505154+0000", 2024-09-12T07:21:58.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.035926362000000003 2024-09-12T07:21:58.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505239+0000", 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5261999999999998e-05 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505330+0000", 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1436999999999999e-05 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.569 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116567 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5312512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.008267270999994, 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036305875000000001, 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116567 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469234+0000", 2024-09-12T07:21:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.58e-07 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469242+0000", 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.61e-06 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469249+0000", 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5549999999999999e-06 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469254+0000", 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9899999999999997e-06 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505361+0000", 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036106923999999999 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505449+0000", 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8531999999999999e-05 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505540+0000", 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0805999999999998e-05 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:116568 3.19 3:9ec14c08:::rbd_data.1160d60c02f0.0000000000000004:15 [read 5353472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.008240598, 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036561383000000003, 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.575 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116568 2024-09-12T07:21:58.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.575 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469261+0000", 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7899999999999999e-07 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469263+0000", 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7549999999999999e-06 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469269+0000", 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7219999999999996e-06 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.469273+0000", 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:58.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505631+0000", 2024-09-12T07:21:58.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036357997000000003 2024-09-12T07:21:58.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505711+0000", 2024-09-12T07:21:58.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9331000000000003e-05 2024-09-12T07:21:58.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:20:26.505822+0000", 2024-09-12T07:21:58.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000111208 2024-09-12T07:21:58.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.579 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.579 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.579 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:58.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:21:58.581 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:58.581+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:58.681 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:58.682+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:58.738 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:58.738 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:58.738 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:21:58.738 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:58.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:21:58.782 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:58.783+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:58.858 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:58.858 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:21:58.858 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:21:58.858 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:21:58.859 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:58.867 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:21:58.911 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:21:58.913+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:21:58.978 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:21:58.978 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-12T07:21:58.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-12T07:21:58.979 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-12T07:21:58.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.979 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169456 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1527808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:58.979 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:58.979 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.458115864, 2024-09-12T07:21:58.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068013471000000006, 2024-09-12T07:21:58.979 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.979 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.980 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169456 2024-09-12T07:21:58.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.980 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:58.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520227+0000", 2024-09-12T07:21:58.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520228+0000", 2024-09-12T07:21:58.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.63e-07 2024-09-12T07:21:58.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520230+0000", 2024-09-12T07:21:58.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3420000000000002e-06 2024-09-12T07:21:58.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520237+0000", 2024-09-12T07:21:58.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5950000000000004e-06 2024-09-12T07:21:58.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520242+0000", 2024-09-12T07:21:58.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:58.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588003+0000", 2024-09-12T07:21:58.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067761181000000004 2024-09-12T07:21:58.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588071+0000", 2024-09-12T07:21:58.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8095000000000004e-05 2024-09-12T07:21:58.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588241+0000", 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016951999999999999 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169457 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1568768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.458094078999999, 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394231, 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169457 2024-09-12T07:21:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520249+0000", 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520250+0000", 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6899999999999997e-07 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520252+0000", 2024-09-12T07:21:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1959999999999998e-06 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520259+0000", 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1509999999999998e-06 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520264+0000", 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588275+0000", 2024-09-12T07:21:58.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068010854999999995 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588350+0000", 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5402999999999996e-05 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588643+0000", 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029306599999999998 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169458 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.458071603, 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069257746999999995, 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169458 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520272+0000", 2024-09-12T07:21:58.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520274+0000", 2024-09-12T07:21:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3350000000000001e-06 2024-09-12T07:21:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520281+0000", 2024-09-12T07:21:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6819999999999997e-06 2024-09-12T07:21:58.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520286+0000", 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0479999999999998e-06 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588740+0000", 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068453571000000005 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.588810+0000", 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0611000000000005e-05 2024-09-12T07:21:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589529+0000", 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071908599999999997 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169459 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1650688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.458049311000003, 2024-09-12T07:21:58.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069503178999999998, 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169459 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:58.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:58.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520294+0000", 2024-09-12T07:21:58.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9299999999999999e-07 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520296+0000", 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1890000000000001e-06 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520303+0000", 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6749999999999996e-06 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520308+0000", 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1240000000000004e-06 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589563+0000", 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069254688999999994 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589637+0000", 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3711e-05 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589797+0000", 2024-09-12T07:21:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016039799999999999 2024-09-12T07:21:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.002 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169460 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1691648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.458028259000002, 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069750706999999995, 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169460 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520315+0000", 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7399999999999999e-07 2024-09-12T07:21:59.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520318+0000", 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2639999999999998e-06 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520324+0000", 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3260000000000001e-06 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520329+0000", 2024-09-12T07:21:59.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6959999999999996e-06 2024-09-12T07:21:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589830+0000", 2024-09-12T07:21:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069501010000000002 2024-09-12T07:21:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.589903+0000", 2024-09-12T07:21:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3558000000000003e-05 2024-09-12T07:21:59.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590066+0000", 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016247899999999999 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169461 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.458007131000002, 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069982339000000005, 2024-09-12T07:21:59.007 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169461 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520336+0000", 2024-09-12T07:21:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520339+0000", 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520345+0000", 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5359999999999998e-06 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520350+0000", 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.079e-06 2024-09-12T07:21:59.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590098+0000", 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069747117999999997 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590168+0000", 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9982999999999997e-05 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590318+0000", 2024-09-12T07:21:59.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015076799999999999 2024-09-12T07:21:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169462 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1773568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.457985016000002, 2024-09-12T07:21:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070210122, 2024-09-12T07:21:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.012 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169462 2024-09-12T07:21:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520358+0000", 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520359+0000", 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9399999999999995e-07 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520361+0000", 2024-09-12T07:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0109999999999999e-06 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520367+0000", 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.285e-06 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520374+0000", 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0899999999999999e-06 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590348+0000", 2024-09-12T07:21:59.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069974233999999996 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590418+0000", 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9314999999999996e-05 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590568+0000", 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000150693 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169463 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.457961122999997, 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070436171000000006, 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169463 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.520382+0000", 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524036+0000", 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036543130000000002 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524043+0000", 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0639999999999996e-06 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524058+0000", 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.411e-05 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524065+0000", 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9780000000000005e-06 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590598+0000", 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.066532540000000001 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590669+0000", 2024-09-12T07:21:59.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0598000000000004e-05 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.590818+0000", 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014956799999999999 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169469 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.454143858999998, 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068176724999999994, 2024-09-12T07:21:59.021 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169469 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524199+0000", 2024-09-12T07:21:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524200+0000", 2024-09-12T07:21:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8000000000000001e-07 2024-09-12T07:21:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524202+0000", 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5100000000000001e-06 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524209+0000", 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524213+0000", 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-12T07:21:59.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592153+0000", 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067939948 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592225+0000", 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2078999999999999e-05 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592376+0000", 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015063000000000001 2024-09-12T07:21:59.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169470 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2101248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.454122838000004, 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068394620000000003, 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169470 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524220+0000", 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524221+0000", 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8700000000000001e-07 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524223+0000", 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1619999999999998e-06 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524230+0000", 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.657e-06 2024-09-12T07:21:59.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524235+0000", 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0799999999999996e-06 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592429+0000", 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068194553000000005 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592512+0000", 2024-09-12T07:21:59.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3123999999999999e-05 2024-09-12T07:21:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592615+0000", 2024-09-12T07:21:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102657 2024-09-12T07:21:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169471 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2142208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.454102053, 2024-09-12T07:21:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068565151000000005, 2024-09-12T07:21:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169471 2024-09-12T07:21:59.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524241+0000", 2024-09-12T07:21:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6300000000000001e-07 2024-09-12T07:21:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524244+0000", 2024-09-12T07:21:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4880000000000001e-06 2024-09-12T07:21:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524250+0000", 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1890000000000004e-06 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524256+0000", 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5230000000000003e-06 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592647+0000", 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068391788999999995 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592719+0000", 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1859000000000004e-05 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592806+0000", 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6940000000000004e-05 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169472 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:59.035 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.454080888999997, 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068745155000000002, 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169472 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:59.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524262+0000", 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524263+0000", 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4499999999999998e-07 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524265+0000", 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.176e-06 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524271+0000", 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4439999999999996e-06 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524276+0000", 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7949999999999998e-06 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592836+0000", 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068560461000000003 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.592907+0000", 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0258000000000004e-05 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593007+0000", 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100676 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169473 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2224128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.454060519000002, 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068911932999999995, 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169473 2024-09-12T07:21:59.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524283+0000", 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.34e-07 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524285+0000", 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0209999999999998e-06 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524291+0000", 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8989999999999998e-06 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524296+0000", 2024-09-12T07:21:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0069999999999997e-06 2024-09-12T07:21:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593038+0000", 2024-09-12T07:21:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068742019000000001 2024-09-12T07:21:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593109+0000", 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1075999999999995e-05 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593195+0000", 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5576999999999997e-05 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169474 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2265088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.044 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.454040325000001, 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069079421000000002, 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169474 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524303+0000", 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3700000000000001e-07 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524305+0000", 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1579999999999999e-06 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524311+0000", 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524316+0000", 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1109999999999999e-06 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593225+0000", 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068909140999999993 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593298+0000", 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2432e-05 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593382+0000", 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4404e-05 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169475 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2306048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.454020413999999, 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069270256000000002, 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169475 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524323+0000", 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7500000000000001e-07 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524325+0000", 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1509999999999998e-06 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524332+0000", 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2650000000000002e-06 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524336+0000", 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9699999999999998e-06 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593418+0000", 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069081988999999996 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593491+0000", 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2619000000000004e-05 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593593+0000", 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101887 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169476 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.453996334000003, 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069449397999999996, 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.053 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169476 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524347+0000", 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4900000000000001e-07 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524350+0000", 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.481e-06 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524356+0000", 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6989999999999997e-06 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524362+0000", 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2320000000000001e-06 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593623+0000", 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069261328999999996 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593706+0000", 2024-09-12T07:21:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2528999999999995e-05 2024-09-12T07:21:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593796+0000", 2024-09-12T07:21:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0679000000000004e-05 2024-09-12T07:21:59.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169477 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2387968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.453973529000002, 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069660978999999998, 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169477 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524370+0000", 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3299999999999998e-07 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524372+0000", 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1519999999999999e-06 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524379+0000", 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.562e-06 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524383+0000", 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5430000000000004e-06 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593852+0000", 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069468641999999997 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.593926+0000", 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4070000000000001e-05 2024-09-12T07:21:59.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594031+0000", 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104677 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169478 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2428928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.453953112999997, 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069830325999999998, 2024-09-12T07:21:59.062 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169478 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524390+0000", 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524393+0000", 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.092e-06 2024-09-12T07:21:59.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524399+0000", 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3539999999999996e-06 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524404+0000", 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9590000000000003e-06 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594062+0000", 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069658017000000003 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594131+0000", 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9226000000000004e-05 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594220+0000", 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9295999999999993e-05 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169479 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 2469888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.453932639000001, 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.069996201999999993, 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169479 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.067 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.524411+0000", 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530070+0000", 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056590099999999999 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530075+0000", 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8200000000000002e-06 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530085+0000", 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530093+0000", 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5519999999999999e-06 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594250+0000", 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.064157642000000001 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594318+0000", 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.737e-05 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.594407+0000", 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9117999999999996e-05 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4448.0:169502 3.2 3:43f2bdeb:::rbd_data.1160559f6e27.0000000000000000:2f [read 3411968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.447477714000001, 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068036163999999996, 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "continuous": false, 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-12T07:21:59.071 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4448", 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 169502 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.530865+0000", 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535987+0000", 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051212549999999999 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.535993+0000", 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0639999999999999e-06 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536003+0000", 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0549e-05 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.536012+0000", 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2309999999999996e-06 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598753+0000", 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.062741665000000002 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598822+0000", 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8689999999999995e-05 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-12T07:21:09.598902+0000", 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9709999999999994e-05 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T07:21:59.075 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:21:59.077 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:21:59.078 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:59.077+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:59.178 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:21:59.179+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:21:59.199 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:59.199 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:59.209 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:21:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:21:59.278 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:59.279+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:59.341 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:59.341 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:59.341 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:59.342 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:59.342 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:59.352 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:21:59.393 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:59.394+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:59.480 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:21:59.480+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:21:59.506 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:59.506 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:21:59.506 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 121.89387600800001, 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.507 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:21:59.509 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.510 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:21:59.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:21:59.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.511 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.511 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:21:59.511 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:21:59.511 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.511 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.511 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.511 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 116.758415752, 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:21:59.513 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:21:59.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:21:59.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:21:59.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.516 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.516 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.516 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:21:59.516 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:21:59.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.517 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:21:59.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:21:59.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.517 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:21:59.517 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:21:59.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.517 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.517 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:21:59.518 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:59.519 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 116.75418863100001, 2024-09-12T07:21:59.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:21:59.519 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.519 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.519 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.519 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.519 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.519 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:59.519 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.519 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:21:59.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.521 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.523 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.523 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:21:59.523 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:21:59.523 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.523 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.523 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.523 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.523 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.523 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:21:59.523 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:59.523 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 116.026437976, 2024-09-12T07:21:59.523 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:21:59.524 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.525 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:21:59.526 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:21:59.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.526 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.526 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.526 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:21:59.526 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:21:59.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.526 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.526 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.526 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 115.937461387, 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:21:59.528 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:21:59.529 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.530 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:21:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:21:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:21:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:21:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:21:59.531 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:21:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:21:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:21:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:21:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:21:59.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:21:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:21:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:21:59.533 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 115.905910057, 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.534 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.535 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:21:59.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:21:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:21:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.537 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:21:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:21:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:21:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:21:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.538 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 115.90270952100001, 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.539 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:21:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:21:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:21:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.541 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:21:59.542 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:59.543 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 115.89946377299999, 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:59.544 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.545 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:21:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:21:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:21:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:21:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:21:59.546 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.547 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 115.897221942, 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.548 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:21:59.549 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.550 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.551 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:21:59.552 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 115.80611473800001, 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.553 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:21:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:21:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:21:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.554 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:21:59.556 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:59.557 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 115.80228665, 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:21:59.558 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.559 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:21:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.561 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 92.820373868999994, 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.562 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:21:59.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:21:59.564 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.565 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 92.808560224999994, 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:21:59.567 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.568 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.569 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:21:59.570 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 92.807224177999998, 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.571 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:21:59.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:21:59.574 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165942 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.306390164, 2024-09-12T07:21:59.575 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93685439599999998, 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165942 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200798+0000", 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.455e-06 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.577 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200802+0000", 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2390000000000004e-06 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200814+0000", 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1816e-05 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200820+0000", 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8760000000000002e-06 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.578 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200899+0000", 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8695999999999993e-05 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200912+0000", 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3545e-05 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137326+0000", 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93641327100000005 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137519+0000", 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000193298 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137651+0000", 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013219999999999999 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165957 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.305778287000003, 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93691399799999997, 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165957 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.581 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201409+0000", 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1999999999999999e-07 2024-09-12T07:21:59.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201412+0000", 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2590000000000001e-06 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201423+0000", 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0547000000000001e-05 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201428+0000", 2024-09-12T07:21:59.583 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.101e-06 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201488+0000", 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9426999999999999e-05 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201498+0000", 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0883999999999999e-05 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.584 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138057+0000", 2024-09-12T07:21:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93655902000000002 2024-09-12T07:21:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138168+0000", 2024-09-12T07:21:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011034199999999999 2024-09-12T07:21:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138322+0000", 2024-09-12T07:21:59.585 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015469799999999999 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165958 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.305747214999997, 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.937070242, 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:59.586 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165958 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:21:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.58e-07 2024-09-12T07:21:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201442+0000", 2024-09-12T07:21:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.719e-06 2024-09-12T07:21:59.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201447+0000", 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4770000000000002e-06 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201450+0000", 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.219e-06 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201530+0000", 2024-09-12T07:21:59.589 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9833000000000006e-05 2024-09-12T07:21:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201538+0000", 2024-09-12T07:21:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.028e-06 2024-09-12T07:21:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138367+0000", 2024-09-12T07:21:59.590 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93682842899999996 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138429+0000", 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1890000000000005e-05 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138510+0000", 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1289e-05 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165973 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.305614589999998, 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93711003900000001, 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165973 2024-09-12T07:21:59.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.593 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201573+0000", 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8599999999999996e-07 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201580+0000", 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3810000000000004e-06 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201590+0000", 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0169999999999999e-05 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.594 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201596+0000", 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4110000000000001e-06 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201645+0000", 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9156999999999997e-05 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201656+0000", 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1112e-05 2024-09-12T07:21:59.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138538+0000", 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93688176899999998 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138595+0000", 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6759000000000003e-05 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138682+0000", 2024-09-12T07:21:59.596 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7293999999999995e-05 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165974 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 51.305576442000003, 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93725339100000005, 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:59.597 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165974 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:21:59.598 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201611+0000", 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201614+0000", 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.757e-06 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.599 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201620+0000", 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.015e-06 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201623+0000", 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6210000000000001e-06 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201679+0000", 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6017999999999998e-05 2024-09-12T07:21:59.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201685+0000", 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2560000000000002e-06 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138699+0000", 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93701424600000005 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138749+0000", 2024-09-12T07:21:59.601 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9951000000000003e-05 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138864+0000", 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115017 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:59.602 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 37.011656481000003, 2024-09-12T07:21:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0249796689999999, 2024-09-12T07:21:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:21:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:21:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:21:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:21:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:21:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:21:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:21:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.603 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:21:59.604 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495532+0000", 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6130000000000001e-06 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495542+0000", 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7990000000000005e-06 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495556+0000", 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4737e-05 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.605 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495567+0000", 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0348e-05 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495628+0000", 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0711999999999997e-05 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495638+0000", 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0482999999999999e-05 2024-09-12T07:21:59.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520066+0000", 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0244278959999999 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520246+0000", 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000179815 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520510+0000", 2024-09-12T07:21:59.607 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026426599999999998 2024-09-12T07:21:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:21:59.608 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:21:59.608 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:59.610 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:21:59.610 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:59.610+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:59.710 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:21:59.710+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:21:59.768 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:59.768 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:59.768 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:21:59.768 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:59.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:21:59.818 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:21:59.819+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:21:59.912 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:21:59.914+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:21:59.943 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:21:59.943 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:21:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:21:59.944 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:21:59.944 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:21:59.954 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:22:00.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:22:00.038 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:22:00.040+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:22:00.111 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:22:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:22:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:22:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:22:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.630143784, 2024-09-12T07:22:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:22:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:22:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:22:00.114 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.630116904, 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.116 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:22:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:22:00.118 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.119 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.63009288400001, 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.120 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.121 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.122 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:22:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:22:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:22:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:22:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.123 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.124 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.125 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.127 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.630065238, 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.128 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:22:00.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.630041776, 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.132 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.133 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:22:00.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:22:00.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.630019249, 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:22:00.136 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:22:00.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:22:00.139 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.140 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.629997446, 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:22:00.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.142 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.143 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.623363666, 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.146 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.147 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:22:00.148 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.149 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.623343658, 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.150 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.616860626, 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.154 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.155 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:22:00.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.616825339, 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:22:00.158 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.159 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.162 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.616802706, 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.163 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:22:00.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.61674865099999, 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.167 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:22:00.169 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:22:00.170 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:22:00.171 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.61672971, 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:22:00.172 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.173 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.174 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.175 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.616717773, 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.176 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.177 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.178 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.179 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.616706158, 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.181 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.184 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:22:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.60969355, 2024-09-12T07:22:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:22:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:22:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.185 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:22:00.186 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.187 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.188 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.60966983500001, 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:00.189 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:22:00.191 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:22:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:22:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:22:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:22:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:22:00.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.193 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.609538978, 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.194 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.197 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 116.64586818799999, 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.198 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:22:00.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:22:00.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:22:00.202 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:22:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:22:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:22:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:00.203 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:00.203 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:00.204 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:22:00.205 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:22:00.205 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:22:00.206 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:22:00.206 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:22:00.206+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:22:00.206 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:22:00.206 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-12T07:22:00.207 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:22:00.207 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:22:00.214 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:22:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:22:00.231 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:22:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:22:00.249 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:22:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:22:00.265 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:22:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:22:00.269 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T07:22:00.270 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T07:22:00.305 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:22:00.307+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:22:00.353 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:00.353 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:00.362 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:22:00.407 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:22:00.409+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:22:00.468 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:00.469 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:00.477 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:22:00.518 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:22:00.519+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:22:00.608 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:22:00.610+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:22:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:00.624 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:00.632 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:22:00.716 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:22:00.718+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:22:00.779 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:00.779 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:00.788 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:22:00.811 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:22:00.813+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:22:00.911 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:22:00.913+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:22:00.936 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:00.936 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:00.945 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:22:01.012 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:22:01.014+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:22:01.090 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:01.091 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:01.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:22:01.113 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:22:01.114+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:22:01.213 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:22:01.215+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:22:01.227 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:01.227 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:01.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:22:01.277 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 6, 7, 3, 4] out_osds: [1, 5] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-12T07:22:01.277 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T07:22:01.277 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-12T07:22:01.277 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-12T07:22:01.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-12T07:22:01.281 INFO:tasks.ceph.osd.2:Started 2024-09-12T07:22:01.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:22:01.291 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:22:01.322 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:22:01.323+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:22:01.392 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:01.392 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:01.402 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:22:01.415 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:22:01.416+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:22:01.427 INFO:teuthology.orchestra.run.smithi046.stderr:no valid command found; 10 closest matches: 2024-09-12T07:22:01.427 INFO:teuthology.orchestra.run.smithi046.stderr:0 2024-09-12T07:22:01.427 INFO:teuthology.orchestra.run.smithi046.stderr:1 2024-09-12T07:22:01.427 INFO:teuthology.orchestra.run.smithi046.stderr:2 2024-09-12T07:22:01.428 INFO:teuthology.orchestra.run.smithi046.stderr:abort 2024-09-12T07:22:01.428 INFO:teuthology.orchestra.run.smithi046.stderr:assert 2024-09-12T07:22:01.428 INFO:teuthology.orchestra.run.smithi046.stderr:config diff 2024-09-12T07:22:01.428 INFO:teuthology.orchestra.run.smithi046.stderr:config diff get 2024-09-12T07:22:01.428 INFO:teuthology.orchestra.run.smithi046.stderr:config get 2024-09-12T07:22:01.428 INFO:teuthology.orchestra.run.smithi046.stderr:config help [] 2024-09-12T07:22:01.428 INFO:teuthology.orchestra.run.smithi046.stderr:config set ... 2024-09-12T07:22:01.428 INFO:teuthology.orchestra.run.smithi046.stderr:admin_socket: invalid command 2024-09-12T07:22:01.430 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T07:22:01.431 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-12T07:22:01.515 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:22:01.517+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:22:01.536 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:01.537 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:01.545 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:22:01.616 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:22:01.617+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:22:01.654 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:01.654 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:01.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:22:01.717 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:22:01.719+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:22:01.770 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:01.770 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:01.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:22:01.823 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:22:01.825+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:22:01.885 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:01.885 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:01.894 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:22:01.918 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:22:01.920+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:22:02.019 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:22:02.021+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:22:02.041 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:02.041 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:02.050 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:22:02.134 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:22:02.136+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:22:02.138 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:22:02.208 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:02.209 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:02.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:22:02.262 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:22:02.264+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:22:02.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T07:22:02.321 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:22:02.323+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:22:02.331 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:02.331 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:02.340 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:22:02.396 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:22:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:22:02.413 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:22:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:22:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-12T07:22:02.422 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:22:02.424+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:22:02.429 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:22:02.429 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:22:02.445 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:22:02.445 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:22:02.450 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:02.450 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:22:02.450 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:22:02.450 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:02.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:22:02.461 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:22:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:22:02.479 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:22:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:22:02.495 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:22:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:22:02.499 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T07:22:02.499 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T07:22:02.523 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:22:02.525+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:22:02.606 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:02.606 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:22:02.606 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:22:02.607 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:22:02.607 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:02.615 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:22:02.658 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:22:02.660+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:22:02.723 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:22:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:22:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:22:02.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:40115 3.14 e37/24 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:head v 37'9, mlcod=34'7)", 2024-09-12T07:22:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:22:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 125.110488366, 2024-09-12T07:22:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023263688000000001, 2024-09-12T07:22:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:22:02.724 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613311+0000", 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.613312+0000", 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.485e-06 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616772+0000", 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003459939 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616800+0000", 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.815e-05 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616820+0000", 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9528999999999999e-05 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.726 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616936+0000", 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115699 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.616963+0000", 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7047999999999999e-05 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636510+0000", 2024-09-12T07:22:02.727 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019547242999999999 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:57.636574+0000", 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4595000000000001e-05 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63194 3.13 e42/24 3:cb2aaaef:::rbd_data.1160f022dd61.0000000000000004:head v 42'62, mlcod=39'60)", 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 119.97502811, 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.033303915000000003, 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748771+0000", 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.748773+0000", 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8419999999999999e-06 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753861+0000", 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0050876879999999999 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753891+0000", 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0593999999999999e-05 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.753904+0000", 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.324e-05 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754000+0000", 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5803000000000004e-05 2024-09-12T07:22:02.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.754028+0000", 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8195e-05 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782022+0000", 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.027994015000000001 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782075+0000", 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2537999999999997e-05 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:63196 3.1 e42/24 3:81998606:::rbd_data.1160f022dd61.0000000000000005:head v 42'54, mlcod=41'52)", 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 119.970800989, 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.029990869999999999, 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.733 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:22:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752998+0000", 2024-09-12T07:22:02.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.752999+0000", 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0900000000000001e-07 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766528+0000", 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013528645000000001 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766540+0000", 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2111e-05 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766545+0000", 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2750000000000003e-06 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766604+0000", 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8893e-05 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.766621+0000", 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7598e-05 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782938+0000", 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.016316908000000001 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:02.782989+0000", 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0730999999999999e-05 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.737 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:67906 3.b e43/33 3:d7fc0add:::rbd_data.1160f022dd61.0000000000000007:head v 43'24, mlcod=43'22)", 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 119.243050334, 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.03317312, 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:22:02.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480749+0000", 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480750+0000", 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6139999999999999e-06 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.739 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489104+0000", 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083537990000000003 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489124+0000", 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9627e-05 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489134+0000", 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0601999999999999e-05 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489234+0000", 2024-09-12T07:22:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9288000000000002e-05 2024-09-12T07:22:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.489256+0000", 2024-09-12T07:22:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2274000000000001e-05 2024-09-12T07:22:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513862+0000", 2024-09-12T07:22:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.024605697999999999 2024-09-12T07:22:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.513922+0000", 2024-09-12T07:22:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0217999999999998e-05 2024-09-12T07:22:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68852 3.1b e43/24 3:dce208f6:::rbd_data.1160f022dd61.000000000000000a:head v 43'41, mlcod=43'39)", 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 119.15407374500001, 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028115325, 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569725+0000", 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.569727+0000", 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5880000000000001e-06 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577621+0000", 2024-09-12T07:22:02.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0078936860000000005 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577645+0000", 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4324e-05 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577668+0000", 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3288000000000001e-05 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577737+0000", 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8768000000000003e-05 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.577759+0000", 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1960999999999999e-05 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597736+0000", 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.019976653 2024-09-12T07:22:02.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.597841+0000", 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010505699999999999 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68856 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'39, mlcod=43'37)", 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 119.12252241500001, 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031945302000000002, 2024-09-12T07:22:02.747 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601277+0000", 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.601279+0000", 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3870000000000002e-06 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604347+0000", 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.003068126 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604374+0000", 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7075000000000001e-05 2024-09-12T07:22:02.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604386+0000", 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2103000000000001e-05 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604475+0000", 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8689000000000004e-05 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604499+0000", 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4355e-05 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633166+0000", 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028666331 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.633222+0000", 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6236000000000002e-05 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.751 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68435 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'172, mlcod=43'170)", 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 119.119321879, 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.039853513, 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:22:02.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604477+0000", 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.604479+0000", 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.271e-06 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607569+0000", 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0030904700000000001 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607590+0000", 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0635000000000001e-05 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607601+0000", 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0949e-05 2024-09-12T07:22:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607677+0000", 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6025999999999993e-05 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607694+0000", 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7458000000000001e-05 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644281+0000", 2024-09-12T07:22:02.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.036587249000000002 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644331+0000", 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9455e-05 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68437 3.f e43/24 3:f2e95411:::rbd_data.1160f022dd61.0000000000000003:head v 43'174, mlcod=43'170)", 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:22:02.756 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 119.116076131, 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.037024132000000001, 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:22:02.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607723+0000", 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607735+0000", 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1238e-05 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607741+0000", 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4919999999999998e-06 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.607750+0000", 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6090000000000001e-06 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613514+0000", 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0057645159999999999 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.613539+0000", 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4686999999999999e-05 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644703+0000", 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.031163539000000001 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.644747+0000", 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4637e-05 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.760 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:68857 3.1d e43/24 3:bde1efb8:::rbd_data.1160f022dd61.000000000000000e:head v 43'41, mlcod=43'37)", 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 119.11383429999999, 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.025112452, 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609965+0000", 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609967+0000", 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4099999999999998e-06 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609980+0000", 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2598e-05 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.609995+0000", 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5099000000000001e-05 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.610005+0000", 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0352e-05 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611537+0000", 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0015317079999999999 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.611563+0000", 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.6095e-05 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635032+0000", 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.023469114999999999 2024-09-12T07:22:02.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.635077+0000", 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5074999999999998e-05 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69431 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'23, mlcod=41'21)", 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 119.022727096, 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022520651999999999, 2024-09-12T07:22:02.765 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701072+0000", 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.701073+0000", 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.345e-06 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703975+0000", 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0029018569999999999 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.703995+0000", 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9607999999999999e-05 2024-09-12T07:22:02.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704005+0000", 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0611999999999999e-05 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704071+0000", 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5271999999999994e-05 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704091+0000", 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0295000000000001e-05 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723540+0000", 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.01944891 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.723593+0000", 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2753000000000001e-05 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.769 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:69432 3.14 e43/24 3:28259b31:::rbd_data.1160f022dd61.000000000000000b:head v 43'25, mlcod=41'21)", 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 119.01889900800001, 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.02035532, 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704900+0000", 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704901+0000", 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1699999999999999e-07 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704905+0000", 2024-09-12T07:22:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0849999999999999e-06 2024-09-12T07:22:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704913+0000", 2024-09-12T07:22:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8179999999999993e-06 2024-09-12T07:22:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.704924+0000", 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0951000000000001e-05 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711050+0000", 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0061256280000000001 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.711075+0000", 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4901000000000001e-05 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725218+0000", 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.014143134 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.725255+0000", 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.7385999999999997e-05 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119434 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'123, mlcod=64'120)", 2024-09-12T07:22:02.774 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.036986227, 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.028392008999999999, 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686813+0000", 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.686814+0000", 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4100000000000001e-06 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696426+0000", 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0096113150000000005 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696443+0000", 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7439999999999999e-05 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696452+0000", 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6780000000000006e-06 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696510+0000", 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8702999999999997e-05 2024-09-12T07:22:02.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.696542+0000", 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.1207999999999997e-05 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715157+0000", 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.018615639999999999 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.715205+0000", 2024-09-12T07:22:02.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7614999999999997e-05 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119431 3.a e65/24 3:51a4f0f1:::rbd_data.116067a48597.0000000000000001:head v 65'156, mlcod=65'153)", 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.025172583, 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.026132137, 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.779 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698627+0000", 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.698628+0000", 2024-09-12T07:22:02.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.623e-06 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702553+0000", 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0039245840000000001 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702566+0000", 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2955999999999999e-05 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702574+0000", 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6640000000000004e-06 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702638+0000", 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3410000000000004e-05 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.702661+0000", 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3606999999999999e-05 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724714+0000", 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022052282999999999 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.724759+0000", 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5009999999999998e-05 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_repop(client.4448.0:119435 3.1 e65/24 3:83815a73:::rbd_data.116067a48597.0000000000000005:head v 65'125, mlcod=64'120)", 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 96.023836536000005, 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.022232314999999999, 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699963+0000", 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699964+0000", 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4020000000000001e-06 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699973+0000", 2024-09-12T07:22:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7739999999999993e-06 2024-09-12T07:22:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699989+0000", 2024-09-12T07:22:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5730999999999999e-05 2024-09-12T07:22:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.699997+0000", 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6920000000000008e-06 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704590+0000", 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0045928729999999999 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.704621+0000", 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0701999999999998e-05 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722152+0000", 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.017530731000000001 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:26.722195+0000", 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3409999999999999e-05 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.789 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165942 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 54.523002521999999, 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93685439599999998, 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165942 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200797+0000", 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200798+0000", 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.455e-06 2024-09-12T07:22:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200802+0000", 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2390000000000004e-06 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200814+0000", 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1816e-05 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200820+0000", 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8760000000000002e-06 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200899+0000", 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8695999999999993e-05 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.200912+0000", 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3545e-05 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.793 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137326+0000", 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93641327100000005 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137519+0000", 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000193298 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.137651+0000", 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013219999999999999 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.794 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165957 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 54.522390645000002, 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93691399799999997, 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:02.795 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165957 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201408+0000", 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201409+0000", 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1999999999999999e-07 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201412+0000", 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.2590000000000001e-06 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201423+0000", 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0547000000000001e-05 2024-09-12T07:22:02.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201428+0000", 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.101e-06 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201488+0000", 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.9426999999999999e-05 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201498+0000", 2024-09-12T07:22:02.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0883999999999999e-05 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138057+0000", 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93655902000000002 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138168+0000", 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011034199999999999 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138322+0000", 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015469799999999999 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165958 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 54.522359573000003, 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.937070242, 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165958 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201440+0000", 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.58e-07 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201442+0000", 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.719e-06 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201447+0000", 2024-09-12T07:22:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4770000000000002e-06 2024-09-12T07:22:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201450+0000", 2024-09-12T07:22:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.219e-06 2024-09-12T07:22:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201530+0000", 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9833000000000006e-05 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201538+0000", 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.028e-06 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138367+0000", 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93682842899999996 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138429+0000", 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1890000000000005e-05 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138510+0000", 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1289e-05 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165973 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 54.522226947999997, 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93711003900000001, 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165973 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201572+0000", 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201573+0000", 2024-09-12T07:22:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8599999999999996e-07 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201580+0000", 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3810000000000004e-06 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201590+0000", 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0169999999999999e-05 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201596+0000", 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4110000000000001e-06 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201645+0000", 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9156999999999997e-05 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201656+0000", 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1112e-05 2024-09-12T07:22:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138538+0000", 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93688176899999998 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138595+0000", 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6759000000000003e-05 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138682+0000", 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7293999999999995e-05 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:165974 3.1d 3:bf54a4cc:::rbd_data.11606fa8d72f.0000000000000009:2e [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 54.522188800000002, 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93725339100000005, 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 165974 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201610+0000", 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201611+0000", 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0999999999999999e-07 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201614+0000", 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.757e-06 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201620+0000", 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.015e-06 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201623+0000", 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6210000000000001e-06 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201679+0000", 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6017999999999998e-05 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:08.201685+0000", 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2560000000000002e-06 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138699+0000", 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.93701424600000005 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138749+0000", 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9951000000000003e-05 2024-09-12T07:22:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:09.138864+0000", 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115017 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.228268839000002, 2024-09-12T07:22:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0249796689999999, 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495530+0000", 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495532+0000", 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6130000000000001e-06 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:02.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495542+0000", 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.7990000000000005e-06 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495556+0000", 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4737e-05 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495567+0000", 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0348e-05 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495628+0000", 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.0711999999999997e-05 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.495638+0000", 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0482999999999999e-05 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520066+0000", 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0244278959999999 2024-09-12T07:22:02.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520246+0000", 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000179815 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:23.520510+0000", 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026426599999999998 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:02.821 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:02.823 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:22:02.823 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:22:02.824+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:22:02.923 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:22:02.925+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:22:02.971 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:22:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:22:02.971 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:02.976 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:22:02.977+0000 7fda19ee4640 -1 Falling back to public interface 2024-09-12T07:22:02.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:22:03.024 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:22:03.026+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:22:03.125 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:22:03.127+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:22:03.126 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:22:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:22:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:22:03.126 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:03.135 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:22:03.225 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:22:03.227+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:22:03.252 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:22:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:22:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:22:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42253 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2059264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:22:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.771526576, 2024-09-12T07:22:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057245902000000001, 2024-09-12T07:22:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42253 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481636+0000", 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481637+0000", 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2399999999999998e-07 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481640+0000", 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5660000000000001e-06 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481646+0000", 2024-09-12T07:22:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.7460000000000003e-06 2024-09-12T07:22:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481651+0000", 2024-09-12T07:22:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:22:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538692+0000", 2024-09-12T07:22:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057040686 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538773+0000", 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1137000000000006e-05 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538882+0000", 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109288 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42255 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2064384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.77149969600001, 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057421396, 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42255 2024-09-12T07:22:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481663+0000", 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481664+0000", 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9900000000000001e-07 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481666+0000", 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3369999999999998e-06 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481673+0000", 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1280000000000002e-06 2024-09-12T07:22:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481680+0000", 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2110000000000003e-06 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538914+0000", 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057234056999999998 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.538989+0000", 2024-09-12T07:22:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5517000000000002e-05 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539085+0000", 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5647000000000001e-05 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42257 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2069504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.77147567599999, 2024-09-12T07:22:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057614348000000003, 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42257 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:22:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481687+0000", 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481688+0000", 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7599999999999997e-07 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481695+0000", 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3710000000000002e-06 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481703+0000", 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3000000000000004e-06 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481708+0000", 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.062e-06 2024-09-12T07:22:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539116+0000", 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057408351000000003 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539199+0000", 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2884000000000001e-05 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539302+0000", 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102904 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42259 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2074624~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.77144803, 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057832931999999997, 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42259 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481715+0000", 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481716+0000", 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5199999999999997e-07 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481718+0000", 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.452e-06 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481726+0000", 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6890000000000004e-06 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481731+0000", 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2039999999999996e-06 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539334+0000", 2024-09-12T07:22:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057602884 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539411+0000", 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6862999999999999e-05 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539548+0000", 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000137288 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42261 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2079744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:22:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.771424568, 2024-09-12T07:22:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058036219, 2024-09-12T07:22:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42261 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481739+0000", 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1399999999999997e-07 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481741+0000", 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1830000000000001e-06 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481748+0000", 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.063e-06 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481753+0000", 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0359999999999997e-06 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539581+0000", 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057827689000000002 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539667+0000", 2024-09-12T07:22:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5804999999999996e-05 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539775+0000", 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000107929 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42264 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2084864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:22:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.771402041, 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058304195000000003, 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42264 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481761+0000", 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5299999999999999e-07 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481764+0000", 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3360000000000002e-06 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481770+0000", 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6549999999999998e-06 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481775+0000", 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9470000000000002e-06 2024-09-12T07:22:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539835+0000", 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058059571999999997 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.539937+0000", 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102188 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540065+0000", 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012804400000000001 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42266 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 2089984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.77138023800001, 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058548246999999998, 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42266 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.481783+0000", 2024-09-12T07:22:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487249+0000", 2024-09-12T07:22:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.005466617 2024-09-12T07:22:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487254+0000", 2024-09-12T07:22:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8849999999999998e-06 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487267+0000", 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2660000000000001e-05 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.487275+0000", 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.6879999999999999e-06 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540097+0000", 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052822583999999999 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540197+0000", 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9733000000000002e-05 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.540331+0000", 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00013407999999999999 2024-09-12T07:22:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42506 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 116736~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.764746458, 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057414510000000002, 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42506 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488417+0000", 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.39e-07 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488419+0000", 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.933e-06 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488425+0000", 2024-09-12T07:22:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.02e-06 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488429+0000", 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4880000000000001e-06 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545642+0000", 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057212407 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545727+0000", 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5265000000000004e-05 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545831+0000", 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104058 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42509 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 121856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.76472645, 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057653293000000001, 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42509 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.488437+0000", 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493932+0000", 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0054950320000000004 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493937+0000", 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0000000000000004e-06 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493946+0000", 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8139999999999995e-06 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.493955+0000", 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0420000000000003e-06 2024-09-12T07:22:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545862+0000", 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051907499000000003 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.545967+0000", 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104888 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.546090+0000", 2024-09-12T07:22:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012301799999999999 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42564 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 265216~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.75824341800001, 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433920999999999, 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42564 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:22:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494920+0000", 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.89e-07 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494922+0000", 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1019999999999999e-06 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494930+0000", 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4760000000000001e-06 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494935+0000", 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5380000000000002e-06 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552151+0000", 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057216329000000003 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552239+0000", 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7140999999999998e-05 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552354+0000", 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011494599999999999 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42566 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 270336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.758208131, 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057656488999999998, 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42566 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494955+0000", 2024-09-12T07:22:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494958+0000", 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.4329999999999998e-06 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494965+0000", 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9879999999999998e-06 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494970+0000", 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3369999999999999e-06 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552404+0000", 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057433814999999999 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552496+0000", 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.2427000000000001e-05 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552611+0000", 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000115029 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42568 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 275456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.758185498, 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057885977999999998, 2024-09-12T07:22:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42568 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494978+0000", 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0600000000000001e-07 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.494994+0000", 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.6362999999999999e-05 2024-09-12T07:22:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495002+0000", 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9480000000000008e-06 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495008+0000", 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.7209999999999999e-06 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552660+0000", 2024-09-12T07:22:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057652183000000003 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552749+0000", 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9268e-05 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552864+0000", 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000114089 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42570 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 280576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.758131443, 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058068373999999999, 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42570 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495032+0000", 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1399999999999997e-07 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495034+0000", 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.1399999999999998e-06 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495041+0000", 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0550000000000002e-06 2024-09-12T07:22:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495045+0000", 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6430000000000001e-06 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552912+0000", 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057866622999999999 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.552995+0000", 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3330000000000003e-05 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553100+0000", 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105169 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42572 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 285696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:22:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.758112502, 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058294726999999998, 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42572 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:22:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495051+0000", 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.72e-07 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495052+0000", 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.336e-06 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495056+0000", 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6480000000000001e-06 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495059+0000", 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.8250000000000001e-06 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553135+0000", 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058075956999999997 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553210+0000", 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.5078000000000002e-05 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553345+0000", 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000135611 2024-09-12T07:22:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42574 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 290816~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:22:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.75810056500001, 2024-09-12T07:22:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058534405999999997, 2024-09-12T07:22:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42574 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495063+0000", 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.5199999999999998e-07 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495064+0000", 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4920000000000001e-06 2024-09-12T07:22:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495068+0000", 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4819999999999999e-06 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495070+0000", 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.7640000000000001e-06 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553398+0000", 2024-09-12T07:22:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058327688000000003 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553486+0000", 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7368999999999996e-05 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553597+0000", 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111359 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42576 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 295936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.75808895, 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.058777821000000001, 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42576 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.495074+0000", 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501293+0000", 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0062193200000000004 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501299+0000", 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1800000000000004e-06 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501321+0000", 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.196e-05 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.501329+0000", 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8119999999999997e-06 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553642+0000", 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.052313063999999999 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553733+0000", 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0840000000000004e-05 2024-09-12T07:22:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.553852+0000", 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000118645 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42628 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 434176~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.751076342, 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.05741458, 2024-09-12T07:22:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42628 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502087+0000", 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.46e-07 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502089+0000", 2024-09-12T07:22:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.092e-06 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502096+0000", 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9589999999999998e-06 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502103+0000", 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.686e-06 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559321+0000", 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0572176 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559414+0000", 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3128999999999996e-05 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559501+0000", 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7668000000000002e-05 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42630 3.1d 3:be6cd675:::rbd_data.1160947fa40f.0000000000000000:6 [read 439296~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.75105262700001, 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057624921000000003, 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42630 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502110+0000", 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502195+0000", 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4519999999999997e-05 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502199+0000", 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.6289999999999999e-06 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502216+0000", 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7580000000000001e-05 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502222+0000", 2024-09-12T07:22:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.173e-06 2024-09-12T07:22:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559543+0000", 2024-09-12T07:22:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057320507999999999 2024-09-12T07:22:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559623+0000", 2024-09-12T07:22:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9647999999999994e-05 2024-09-12T07:22:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559735+0000", 2024-09-12T07:22:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011286299999999999 2024-09-12T07:22:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42631 3.1d 3.ae6b367d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 123.75092177000001, 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.057541931999999997, 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42631 2024-09-12T07:22:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.502241+0000", 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507876+0000", 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0056343010000000004 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507881+0000", 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2549999999999997e-06 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507895+0000", 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.3848e-05 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.507902+0000", 2024-09-12T07:22:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7710000000000006e-06 2024-09-12T07:22:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559773+0000", 2024-09-12T07:22:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.051870404000000002 2024-09-12T07:22:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.559783+0000", 2024-09-12T07:22:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0353000000000001e-05 2024-09-12T07:22:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67961 3.c 3:30cc6a33:::rbd_data.1160f022dd61.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:22:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 119.78725098, 2024-09-12T07:22:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.061888749, 2024-09-12T07:22:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67961 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465912+0000", 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.465913+0000", 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5199999999999995e-07 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479774+0000", 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.013860832999999999 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479808+0000", 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4255000000000003e-05 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479819+0000", 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0407e-05 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479910+0000", 2024-09-12T07:22:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1748000000000003e-05 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.479995+0000", 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4690000000000004e-05 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for subops from 6", 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.480331+0000", 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033641399999999999 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "sub_op_commit_rec", 2024-09-12T07:22:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.525707+0000", 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.045376088000000002 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "op_commit", 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527726+0000", 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0020186510000000002 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "commit_sent", 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527766+0000", 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9402e-05 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.527801+0000", 2024-09-12T07:22:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5308999999999997e-05 2024-09-12T07:22:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.347 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:03.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:22:03.349 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:22:03.350+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:22:03.449 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:22:03.451+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:22:03.456 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:22:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:22:03.456 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:03.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:22:03.550 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:22:03.552+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:22:03.571 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:22:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:22:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:22:03.572 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:03.580 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:22:03.664 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:22:03.666+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:22:03.729 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:22:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:22:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:22:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:2820 3.1e 3:7b03acb7:::rbd_header.116075042ea1:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T07:22:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:22:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 131.20000573499999, 2024-09-12T07:22:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96956314899999996, 2024-09-12T07:22:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2820 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529709+0000", 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529710+0000", 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2419999999999999e-06 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529715+0000", 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1159999999999998e-06 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529727+0000", 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1804000000000001e-05 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529735+0000", 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.1380000000000007e-06 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529787+0000", 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1725999999999999e-05 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:52.529794+0000", 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.545e-06 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498726+0000", 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.96893196599999998 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.498939+0000", 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00021348299999999999 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:53.499272+0000", 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033312899999999999 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42350 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.34820806899999, 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055186550000000001, 2024-09-12T07:22:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42350 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381507+0000", 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381594+0000", 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.7441000000000005e-05 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381598+0000", 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8600000000000003e-06 2024-09-12T07:22:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381608+0000", 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9110000000000005e-06 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381615+0000", 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1119999999999998e-06 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436527+0000", 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054911736000000003 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436590+0000", 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3050000000000001e-05 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436693+0000", 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010344 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42352 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.34808479199999, 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055323065999999997, 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42352 2024-09-12T07:22:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381630+0000", 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381804+0000", 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017384400000000001 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381807+0000", 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.489e-06 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381816+0000", 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4509999999999992e-06 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381820+0000", 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.617e-06 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436724+0000", 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054903345999999999 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436829+0000", 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000105259 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436953+0000", 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012406 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42354 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 212992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.347881401, 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055345949999999998, 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42354 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381833+0000", 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381938+0000", 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010479700000000001 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381942+0000", 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0960000000000003e-06 2024-09-12T07:22:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381951+0000", 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8070000000000006e-06 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381958+0000", 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5429999999999999e-06 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.436997+0000", 2024-09-12T07:22:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039418999999999 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437080+0000", 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2782999999999997e-05 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437179+0000", 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9505000000000004e-05 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42356 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:22:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.347745574, 2024-09-12T07:22:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055470179000000001, 2024-09-12T07:22:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42356 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.381969+0000", 2024-09-12T07:22:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382219+0000", 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00024970299999999999 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382223+0000", 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4109999999999998e-06 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382234+0000", 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0402e-05 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382241+0000", 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4279999999999999e-06 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437226+0000", 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054984600000000002 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437328+0000", 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010218299999999999 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437439+0000", 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000111452 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42358 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 223232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.347418918, 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055413934999999997, 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42358 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382296+0000", 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0100000000000004e-07 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382301+0000", 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0289999999999999e-06 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382315+0000", 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.4046000000000001e-05 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382323+0000", 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0320000000000003e-06 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437502+0000", 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055179021000000002 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437593+0000", 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1722e-05 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437710+0000", 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000116384 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42360 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 228352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.34738003, 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055556451999999999, 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42360 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382335+0000", 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382669+0000", 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033435399999999998 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382673+0000", 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4220000000000002e-06 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382684+0000", 2024-09-12T07:22:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0263e-05 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382690+0000", 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4459999999999998e-06 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437761+0000", 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055071042000000001 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437817+0000", 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.6094000000000002e-05 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437891+0000", 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3831000000000005e-05 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42361 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.347013156, 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055420807000000002, 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42361 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382702+0000", 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382806+0000", 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000104755 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382810+0000", 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0489999999999997e-06 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382820+0000", 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.431e-06 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382826+0000", 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.437926+0000", 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055100403999999999 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438018+0000", 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1588999999999993e-05 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438122+0000", 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00010437000000000001 2024-09-12T07:22:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42363 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.3468783, 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055513598999999997, 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42363 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.382836+0000", 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383171+0000", 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00033434300000000002 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383175+0000", 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.0260000000000001e-06 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383184+0000", 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.1810000000000007e-06 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383191+0000", 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5930000000000002e-06 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438171+0000", 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054980249000000002 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438254+0000", 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2855999999999994e-05 2024-09-12T07:22:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438350+0000", 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.6351e-05 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42365 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.34650840899999, 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055384254000000001, 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42365 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383206+0000", 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383366+0000", 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00015969500000000001 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383370+0000", 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8890000000000004e-06 2024-09-12T07:22:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383379+0000", 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5759999999999995e-06 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383386+0000", 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4309999999999999e-06 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438392+0000", 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055005866 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438477+0000", 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5556999999999994e-05 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438591+0000", 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011323999999999999 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42367 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.346318211, 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055397870000000002, 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42367 2024-09-12T07:22:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383396+0000", 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383572+0000", 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00017585300000000001 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383576+0000", 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9260000000000002e-06 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383599+0000", 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3053000000000001e-05 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383606+0000", 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5039999999999999e-06 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438622+0000", 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055016155999999997 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438694+0000", 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2261000000000005e-05 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438794+0000", 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000100117 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42369 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.34609813199999, 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055362635, 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42369 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383617+0000", 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383730+0000", 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00011342599999999999 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383733+0000", 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4589999999999998e-06 2024-09-12T07:22:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383743+0000", 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3049999999999998e-06 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383749+0000", 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1240000000000002e-06 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438831+0000", 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055081873000000003 2024-09-12T07:22:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438915+0000", 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4429000000000001e-05 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.438979+0000", 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4018999999999998e-05 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42371 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.345954467, 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055376126999999997, 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42371 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:22:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383760+0000", 2024-09-12T07:22:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383955+0000", 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00019468799999999999 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383959+0000", 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8999999999999999e-06 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383968+0000", 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.4290000000000007e-06 2024-09-12T07:22:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383976+0000", 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4240000000000004e-06 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439015+0000", 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055039058000000002 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439056+0000", 2024-09-12T07:22:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.1514999999999998e-05 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439136+0000", 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0112999999999997e-05 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42373 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 264192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.345728295, 2024-09-12T07:22:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055296555999999997, 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42373 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.383986+0000", 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384156+0000", 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00016923799999999999 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384160+0000", 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9169999999999999e-06 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384170+0000", 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9510000000000001e-06 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384176+0000", 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3740000000000003e-06 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439179+0000", 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055003415999999999 2024-09-12T07:22:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439229+0000", 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9518999999999998e-05 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439283+0000", 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4141000000000002e-05 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42375 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.345527937, 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055309412000000002, 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42375 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384187+0000", 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384471+0000", 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00028401199999999999 2024-09-12T07:22:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384475+0000", 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5920000000000002e-06 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384487+0000", 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1583000000000001e-05 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384494+0000", 2024-09-12T07:22:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.0859999999999996e-06 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439314+0000", 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054820160999999999 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439394+0000", 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.9769000000000001e-05 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439496+0000", 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000102209 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42379 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.34512005400001, 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055279325999999997, 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42379 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384595+0000", 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384960+0000", 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000365273 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384964+0000", 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5789999999999997e-06 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384977+0000", 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2619000000000001e-05 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.384985+0000", 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4100000000000002e-06 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439713+0000", 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054728783000000003 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439765+0000", 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2040000000000003e-05 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.439874+0000", 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000108622 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42383 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.34465417200001, 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055227778999999998, 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42383 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385061+0000", 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385366+0000", 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030530599999999998 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385370+0000", 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5839999999999996e-06 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385383+0000", 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2325e-05 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.385390+0000", 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4449999999999999e-06 2024-09-12T07:22:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440135+0000", 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054744398 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440201+0000", 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6884000000000005e-05 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.440288+0000", 2024-09-12T07:22:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6836999999999995e-05 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42397 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.343051962, 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055265720999999997, 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42397 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.386663+0000", 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387126+0000", 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00046307299999999999 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387130+0000", 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6280000000000004e-06 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387142+0000", 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1848000000000001e-05 2024-09-12T07:22:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387150+0000", 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2309999999999999e-06 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441690+0000", 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054540735 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441815+0000", 2024-09-12T07:22:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000124869 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.441928+0000", 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000113337 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:42401 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 335872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.342517927, 2024-09-12T07:22:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055181719999999997, 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 42401 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387197+0000", 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387797+0000", 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00060028499999999999 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387802+0000", 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.955e-06 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387812+0000", 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.9639999999999998e-06 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.387819+0000", 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.9299999999999997e-06 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442199+0000", 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054379865999999999 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442280+0000", 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0747999999999993e-05 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.442378+0000", 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.8972e-05 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:46526 3.1e 3:7a5f3be8:::rbd_data.1160947fa40f.0000000000000001:6 [read 3299328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 124.08631985700001, 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.055297087000000002, 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 46526 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643395+0000", 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643597+0000", 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00020251199999999999 2024-09-12T07:22:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643602+0000", 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3630000000000004e-06 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643614+0000", 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.2626000000000001e-05 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.643622+0000", 2024-09-12T07:22:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.4540000000000001e-06 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698478+0000", 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.054855946000000003 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698607+0000", 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00012934000000000001 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:19:59.698692+0000", 2024-09-12T07:22:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4845999999999994e-05 2024-09-12T07:22:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:03.819 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:03.821 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:22:03.821 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:22:03.822+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:22:03.922 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:22:03.923+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:22:03.968 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:22:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "num_ops": 0 2024-09-12T07:22:03.968 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:03.976 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:22:04.023 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:22:04.024+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:22:04.082 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [], 2024-09-12T07:22:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "complaint_time": 30, 2024-09-12T07:22:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "num_blocked_ops": 0 2024-09-12T07:22:04.083 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:04.091 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:22:04.135 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:22:04.137+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:22:04.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T07:22:04.207 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T07:22:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2024-09-12T07:22:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2024-09-12T07:22:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2024-09-12T07:22:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67962 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3358720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:22:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.658584142, 2024-09-12T07:22:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10357259100000001, 2024-09-12T07:22:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67962 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549083+0000", 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549084+0000", 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.2099999999999997e-07 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549086+0000", 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.849e-06 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549092+0000", 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3099999999999997e-06 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549097+0000", 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1229999999999999e-06 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652308+0000", 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103210449 2024-09-12T07:22:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652390+0000", 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2204000000000001e-05 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652656+0000", 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00026613500000000003 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67964 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3833856~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.658563605, 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10373825, 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67964 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549104+0000", 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.5599999999999995e-07 2024-09-12T07:22:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549106+0000", 2024-09-12T07:22:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8810000000000001e-06 2024-09-12T07:22:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549112+0000", 2024-09-12T07:22:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1770000000000003e-06 2024-09-12T07:22:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549117+0000", 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9309999999999999e-06 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652687+0000", 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103569694 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652758+0000", 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1200999999999998e-05 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652842+0000", 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3809999999999999e-05 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67965 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:22:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.65854360199999, 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10392061800000001, 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67965 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549124+0000", 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7100000000000002e-07 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549127+0000", 2024-09-12T07:22:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2170000000000001e-06 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549133+0000", 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2090000000000002e-06 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549137+0000", 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7360000000000001e-06 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652878+0000", 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103740973 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.652951+0000", 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2446000000000004e-05 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653044+0000", 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.3566000000000005e-05 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67966 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:22:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.658523653, 2024-09-12T07:22:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104366764, 2024-09-12T07:22:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67966 2024-09-12T07:22:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:22:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549144+0000", 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8999999999999997e-07 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549146+0000", 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.069e-06 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549153+0000", 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2439999999999998e-06 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549158+0000", 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.1959999999999999e-06 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653074+0000", 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10391655 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653145+0000", 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1052999999999999e-05 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653511+0000", 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036516199999999998 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67967 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 2375680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.658502785, 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10477946, 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67967 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549165+0000", 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.96e-07 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549167+0000", 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0260000000000002e-06 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549175+0000", 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7589999999999997e-06 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549180+0000", 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9540000000000003e-06 2024-09-12T07:22:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653554+0000", 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10437445200000001 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653627+0000", 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.3065000000000006e-05 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653944+0000", 2024-09-12T07:22:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00031680800000000002 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67977 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4636672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.658481084, 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10493018599999999, 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67977 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549186+0000", 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549187+0000", 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5900000000000003e-07 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549189+0000", 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9640000000000002e-06 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549195+0000", 2024-09-12T07:22:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.088e-06 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549200+0000", 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8570000000000003e-06 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.653978+0000", 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104778369 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654063+0000", 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5016000000000001e-05 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654117+0000", 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.3532999999999997e-05 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67979 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 4620288~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.658460644, 2024-09-12T07:22:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105030238, 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67979 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:22:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549207+0000", 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.01e-07 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549209+0000", 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9989999999999998e-06 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549216+0000", 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8020000000000003e-06 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549221+0000", 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.826e-06 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654152+0000", 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104931564 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654195+0000", 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2314999999999997e-05 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654237+0000", 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.2330999999999998e-05 2024-09-12T07:22:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67981 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5128192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.65843292300001, 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10521238400000001, 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67981 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549235+0000", 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.4799999999999999e-07 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549237+0000", 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.367e-06 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549245+0000", 2024-09-12T07:22:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.2540000000000003e-06 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549250+0000", 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4389999999999999e-06 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654278+0000", 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105028078 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654360+0000", 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2210000000000001e-05 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654447+0000", 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6588000000000005e-05 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67983 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5603328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.658409988, 2024-09-12T07:22:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10541062599999999, 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67983 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549257+0000", 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549258+0000", 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.0499999999999999e-07 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549260+0000", 2024-09-12T07:22:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0080000000000001e-06 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549266+0000", 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.3380000000000001e-06 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549271+0000", 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9910000000000002e-06 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654496+0000", 2024-09-12T07:22:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105224437 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654584+0000", 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.8504999999999994e-05 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654668+0000", 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4042000000000006e-05 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67985 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5619712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.658389688, 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10559758299999999, 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67985 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549278+0000", 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5999999999999999e-07 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549280+0000", 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9759999999999998e-06 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549287+0000", 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.4069999999999998e-06 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549291+0000", 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.7620000000000003e-06 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654712+0000", 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105421028 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654790+0000", 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.7448999999999997e-05 2024-09-12T07:22:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654875+0000", 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.5500999999999993e-05 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67987 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.658369412, 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105746875, 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67987 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549298+0000", 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.9799999999999999e-07 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549300+0000", 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.0190000000000001e-06 2024-09-12T07:22:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549307+0000", 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.331e-06 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549312+0000", 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9350000000000002e-06 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.654918+0000", 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10560652299999999 2024-09-12T07:22:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655001+0000", 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2960999999999994e-05 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655045+0000", 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.3708000000000002e-05 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67988 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6111232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.658349253, 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105859043, 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67988 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549318+0000", 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549428+0000", 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000109813 2024-09-12T07:22:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549431+0000", 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.4400000000000001e-06 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549439+0000", 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.909e-06 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549445+0000", 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8730000000000004e-06 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655076+0000", 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105631262 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655131+0000", 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4573000000000001e-05 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655177+0000", 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6173e-05 2024-09-12T07:22:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67993 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6389760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.65821205500001, 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.105932202, 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67993 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549455+0000", 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549456+0000", 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.0699999999999997e-07 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549458+0000", 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.2479999999999999e-06 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549465+0000", 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.8000000000000001e-06 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549470+0000", 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.4079999999999997e-06 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655220+0000", 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10574973999999999 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655304+0000", 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4274e-05 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655388+0000", 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.3225000000000004e-05 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:67994 3.14 3:2c3c0711:::rbd_data.116052cf9e71.0000000000000002:9 [read 6406144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.65818940699999, 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10604134799999999, 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 67994 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.549478+0000", 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.554993+0000", 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0055148460000000003 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555003+0000", 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0421000000000001e-05 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555020+0000", 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.7119999999999999e-05 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555031+0000", 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0774000000000001e-05 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655425+0000", 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.100393808 2024-09-12T07:22:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655473+0000", 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8417000000000001e-05 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.655519+0000", 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.5961999999999998e-05 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68090 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 2359296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.651727421, 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10382216399999999, 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68090 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555940+0000", 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.8200000000000001e-07 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555942+0000", 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.8899999999999999e-06 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555948+0000", 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8749999999999997e-06 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.555953+0000", 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.6439999999999999e-06 2024-09-12T07:22:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659429+0000", 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10347603900000001 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659507+0000", 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.8518999999999998e-05 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659762+0000", 2024-09-12T07:22:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00025481499999999998 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68096 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 98304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.651497784, 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.104086679, 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68096 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.556170+0000", 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562107+0000", 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0059371329999999998 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562113+0000", 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 5.8540000000000002e-06 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562124+0000", 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1576000000000001e-05 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.562132+0000", 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.0620000000000001e-06 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659806+0000", 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.097674059999999993 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.659889+0000", 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.2214999999999998e-05 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.660256+0000", 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00036777899999999999 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68142 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6111232~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:22:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.644574093, 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10359112500000001, 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68142 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:22:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563093+0000", 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563094+0000", 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.34e-07 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.280 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563096+0000", 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 2.3429999999999998e-06 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563105+0000", 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.0219999999999993e-06 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563112+0000", 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.5339999999999996e-06 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.281 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666298+0000", 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103186339 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666384+0000", 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.6434999999999995e-05 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666684+0000", 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030001799999999999 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.282 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68146 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6193152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.64421451, 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10368026800000001, 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68146 2024-09-12T07:22:04.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563453+0000", 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.284 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563454+0000", 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 9.5799999999999998e-07 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563457+0000", 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.535e-06 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563470+0000", 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.269e-05 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.285 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563477+0000", 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.6649999999999997e-06 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666735+0000", 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103258208 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.666824+0000", 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.9435e-05 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.287 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.287 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667133+0000", 2024-09-12T07:22:04.287 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.00030877699999999999 2024-09-12T07:22:04.287 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.287 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.287 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.287 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:68150 3.14 3:2d3ef1a7:::rbd_data.116063d335b3.0000000000000002:8 [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-12T07:22:04.287 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:22:04.287 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 120.64413452399999, 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.103964401, 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "started", 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 68150 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:22:04.288 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563533+0000", 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 3.5600000000000001e-07 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.289 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563535+0000", 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563541+0000", 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.1399999999999997e-06 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.563546+0000", 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.8790000000000002e-06 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667158+0000", 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.10361177000000001 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "started", 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667229+0000", 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1276e-05 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:20:03.667497+0000", 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.000268027 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.291 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "osd_op(client.4448.0:201405 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 42.714787272999999, 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0005192119999999, 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: "continuous": false, 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "reached pg", 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.4448", 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: "client_addr": "172.21.15.46:0/2167487566", 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 201405 2024-09-12T07:22:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated", 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read", 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492880+0000", 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled", 2024-09-12T07:22:04.293 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492882+0000", 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.9530000000000002e-06 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read", 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492888+0000", 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 6.2060000000000004e-06 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched", 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492904+0000", 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.5628e-05 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.294 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "queued_for_pg", 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492912+0000", 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 8.4039999999999995e-06 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492962+0000", 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 4.9425999999999997e-05 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "waiting for peered", 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:21.492969+0000", 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 7.1729999999999998e-06 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.295 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "reached_pg", 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.493388+0000", 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.0004188510000001 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done", 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2024-09-12T07:21:22.493399+0000", 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 1.1571e-05 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T07:22:04.296 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T07:22:04.298 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:22:04.299+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:22:04.298 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T07:22:04.399 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:22:04.400+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:22:04.407 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:22:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:22:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:22:04.407 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:22:04.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T07:22:04.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T07:22:04.500 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:22:04.501+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:22:04.525 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:22:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:22:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-12T07:22:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-12T07:22:04.526 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:22:04.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T07:22:04.619 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:22:04.620+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:22:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0: {} 2024-09-12T07:22:04.625 INFO:teuthology.orchestra.run.smithi046.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T07:22:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1: {} 2024-09-12T07:22:04.642 INFO:teuthology.orchestra.run.smithi046.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T07:22:04.642 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-12T07:22:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3: {} 2024-09-12T07:22:04.659 INFO:teuthology.orchestra.run.smithi046.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T07:22:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4: {} 2024-09-12T07:22:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T07:22:04.684 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:04.684 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:04.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T07:22:04.694 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5: {} 2024-09-12T07:22:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T07:22:04.701 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:22:04.702+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:22:04.710 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6: {} 2024-09-12T07:22:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T07:22:04.727 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7: {} 2024-09-12T07:22:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T07:22:04.730 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T07:22:04.731 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T07:22:04.802 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:22:04.803+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:22:04.817 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:04.817 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:04.826 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T07:22:04.903 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:22:04.905+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:22:04.934 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:04.934 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:04.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T07:22:05.005 INFO:tasks.ceph.osd.7.smithi064.stderr:2024-09-12T07:22:05.006+0000 7f9049e6f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33842) UID: 0 2024-09-12T07:22:05.090 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:05.091 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:05.100 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T07:22:05.105 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:22:05.107+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:22:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is set 2024-09-12T07:22:05.206 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:22:05.207+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:22:05.215 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:05.215 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:05.224 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T07:22:05.308 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:22:05.310+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:22:05.373 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:05.373 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:05.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T07:22:05.424 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:22:05.425+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:22:05.508 INFO:tasks.ceph.osd.5.smithi064.stderr:2024-09-12T07:22:05.510+0000 7f595c460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33836) UID: 0 2024-09-12T07:22:05.531 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:05.531 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:05.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T07:22:05.610 INFO:tasks.ceph.osd.6.smithi064.stderr:2024-09-12T07:22:05.611+0000 7f0cb51d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33841) UID: 0 2024-09-12T07:22:05.651 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:05.652 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:05.660 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T07:22:05.744 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:22:05.746+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:22:05.808 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:05.808 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:05.811 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-12T07:22:05.813+0000 7f29277af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33609) UID: 0 2024-09-12T07:22:05.817 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T07:22:05.912 INFO:tasks.ceph.osd.4.smithi064.stderr:2024-09-12T07:22:05.914+0000 7f0dde824640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33834) UID: 0 2024-09-12T07:22:05.964 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:05.965 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:05.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T07:22:06.012 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-12T07:22:06.014+0000 7f1064f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33621) UID: 0 2024-09-12T07:22:06.113 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:22:06.115+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:22:06.120 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:06.120 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:06.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T07:22:06.214 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-12T07:22:06.216+0000 7f63bbf3b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33606) UID: 0 2024-09-12T07:22:06.248 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:06.248 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:06.251 INFO:teuthology.orchestra.run.smithi046.stdout:All operations completed A-OK! 2024-09-12T07:22:06.257 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T07:22:06.263 DEBUG:teuthology.parallel:result is None 2024-09-12T07:22:06.263 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-12T07:22:06.272 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-12T07:22:06.311 INFO:tasks.thrashosds:joining thrashosds 2024-09-12T07:22:06.364 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:06.364 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:06.373 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T07:22:06.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:22:06.480 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:06.481 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:06.489 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T07:22:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:no valid command found; 10 closest matches: 2024-09-12T07:22:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:0 2024-09-12T07:22:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:1 2024-09-12T07:22:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2 2024-09-12T07:22:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:abort 2024-09-12T07:22:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:assert 2024-09-12T07:22:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:config diff 2024-09-12T07:22:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:config diff get 2024-09-12T07:22:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:config get 2024-09-12T07:22:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:config help [] 2024-09-12T07:22:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:config set ... 2024-09-12T07:22:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:admin_socket: invalid command 2024-09-12T07:22:06.537 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T07:22:06.537 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-12T07:22:06.596 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:06.597 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:06.605 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T07:22:06.713 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:06.713 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:06.721 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T07:22:06.828 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:06.829 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:06.837 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T07:22:06.943 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:06.943 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:06.952 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T07:22:07.058 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR: (22) Invalid argument 2024-09-12T07:22:07.058 INFO:teuthology.orchestra.run.smithi064.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T07:22:07.100 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:22:07.101+0000 7fda19ee4640 -1 osd.2 140 log_to_monitors true 2024-09-12T07:22:07.173 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T07:22:07.193 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-12T07:22:07.194+0000 7fda0dc92640 -1 osd.2 164 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T07:22:08.169 INFO:teuthology.orchestra.run.smithi046.stderr:noscrub is unset 2024-09-12T07:22:08.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T07:22:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:nodeep-scrub is unset 2024-09-12T07:22:11.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T07:22:11.660 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:22:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-12T07:22:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-12T07:22:11.660 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:22:16.670 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-12T07:22:16.670 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-12T07:22:16.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-12T07:22:16.992 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-12T07:22:16.993+0000 7f0757424640 -1 mon.a@0(leader).osd e168 definitely_dead 0 2024-09-12T07:22:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:marked in osd.1. 2024-09-12T07:22:17.164 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-12T07:22:17.164 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-12T07:22:17.164 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-12T07:22:17.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-12T07:22:17.472 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-12T07:22:17.473+0000 7f0757424640 -1 mon.a@0(leader).osd e169 definitely_dead 0 2024-09-12T07:22:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:marked in osd.5. 2024-09-12T07:22:18.183 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-12T07:22:18.183 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-12T07:22:18.306 INFO:teuthology.orchestra.run.smithi046.stdout:mon.a: {} 2024-09-12T07:22:18.306 INFO:teuthology.orchestra.run.smithi046.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-12T07:22:18.315 INFO:teuthology.orchestra.run.smithi046.stdout:mon.b: {} 2024-09-12T07:22:18.316 INFO:teuthology.orchestra.run.smithi046.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-12T07:22:18.324 INFO:teuthology.orchestra.run.smithi046.stdout:mon.c: {} 2024-09-12T07:22:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-12T07:22:18.335 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-12T07:22:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.0 to 1 (10000) 2024-09-12T07:22:19.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-12T07:22:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-12T07:22:20.162 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-12T07:22:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.1 to 1 (10000) 2024-09-12T07:22:21.163 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-12T07:22:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-12T07:22:22.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-12T07:22:23.185 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.3 to 1 (10000) 2024-09-12T07:22:23.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-12T07:22:24.196 INFO:teuthology.orchestra.run.smithi046.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-12T07:22:24.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-12T07:22:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.4 to 1 (10000) 2024-09-12T07:22:25.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-12T07:22:26.204 INFO:teuthology.orchestra.run.smithi046.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-12T07:22:26.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-12T07:22:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.5 to 1 (10000) 2024-09-12T07:22:27.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-12T07:22:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-12T07:22:28.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-12T07:22:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.6 to 1 (10000) 2024-09-12T07:22:29.232 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-12T07:22:30.224 INFO:teuthology.orchestra.run.smithi046.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-12T07:22:30.242 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-12T07:22:31.237 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.7 to 1 (10000) 2024-09-12T07:22:31.253 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-12T07:22:32.241 INFO:teuthology.orchestra.run.smithi046.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-12T07:22:32.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-12T07:22:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:reweighted osd.2 to 1 (10000) 2024-09-12T07:22:33.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-12T07:22:34.249 INFO:teuthology.orchestra.run.smithi046.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-12T07:22:34.267 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-12T07:22:34.267 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-12T07:22:34.267 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-12T07:22:34.267 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-12T07:22:34.267 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-12T07:22:34.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T07:22:34.576 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:22:34.576 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":186,"fsid":"8412ed58-ac5e-4549-b523-1a58cf9f9429","created":"2024-09-12T07:19:10.328057+0000","modified":"2024-09-12T07:22:34.246048+0000","last_up_change":"2024-09-12T07:22:08.169187+0000","last_in_change":"2024-09-12T07:22:18.153362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T07:19:23.507151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T07:19:28.593632+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-12T07:19:40.555421+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":163,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"710c25c3-92d3-4b51-a7fc-670879faa69c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6817","nonce":4081287959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6819","nonce":4081287959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6823","nonce":4081287959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6821","nonce":4081287959}]},"public_addr":"172.21.15.46:6817/4081287959","cluster_addr":"172.21.15.46:6819/4081287959","heartbeat_back_addr":"172.21.15.46:6823/4081287959","heartbeat_front_addr":"172.21.15.46:6821/4081287959","state":["exists","up"]},{"osd":1,"uuid":"04479191-c52e-4b60-aa0d-3e911f869062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6825","nonce":3980488213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6827","nonce":3980488213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6831","nonce":3980488213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6829","nonce":3980488213}]},"public_addr":"172.21.15.46:6825/3980488213","cluster_addr":"172.21.15.46:6827/3980488213","heartbeat_back_addr":"172.21.15.46:6831/3980488213","heartbeat_front_addr":"172.21.15.46:6829/3980488213","state":["exists","up"]},{"osd":2,"uuid":"ba673e51-96aa-424b-a4df-26432d633ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":140,"up_from":165,"up_thru":172,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3239477423},{"type":"v1","addr":"172.21.15.46:6801","nonce":3239477423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3239477423},{"type":"v1","addr":"172.21.15.46:6803","nonce":3239477423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3239477423},{"type":"v1","addr":"172.21.15.46:6807","nonce":3239477423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3239477423},{"type":"v1","addr":"172.21.15.46:6805","nonce":3239477423}]},"public_addr":"172.21.15.46:6801/3239477423","cluster_addr":"172.21.15.46:6803/3239477423","heartbeat_back_addr":"172.21.15.46:6807/3239477423","heartbeat_front_addr":"172.21.15.46:6805/3239477423","state":["exists","up"]},{"osd":3,"uuid":"c516e59c-834c-4fb0-9ff8-bcc80b21d28a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6809","nonce":3848260327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6811","nonce":3848260327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6815","nonce":3848260327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6813","nonce":3848260327}]},"public_addr":"172.21.15.46:6809/3848260327","cluster_addr":"172.21.15.46:6811/3848260327","heartbeat_back_addr":"172.21.15.46:6815/3848260327","heartbeat_front_addr":"172.21.15.46:6813/3848260327","state":["exists","up"]},{"osd":4,"uuid":"450a70cd-ebb2-48ed-a766-184f58e7032e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6817","nonce":3464256744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6819","nonce":3464256744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6823","nonce":3464256744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6821","nonce":3464256744}]},"public_addr":"172.21.15.64:6817/3464256744","cluster_addr":"172.21.15.64:6819/3464256744","heartbeat_back_addr":"172.21.15.64:6823/3464256744","heartbeat_front_addr":"172.21.15.64:6821/3464256744","state":["exists","up"]},{"osd":5,"uuid":"a4efa333-bf57-45e7-a281-1d3bc9d2e82b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6825","nonce":1789414693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6827","nonce":1789414693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6831","nonce":1789414693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6829","nonce":1789414693}]},"public_addr":"172.21.15.64:6825/1789414693","cluster_addr":"172.21.15.64:6827/1789414693","heartbeat_back_addr":"172.21.15.64:6831/1789414693","heartbeat_front_addr":"172.21.15.64:6829/1789414693","state":["exists","up"]},{"osd":6,"uuid":"bcaad352-f9ee-426c-bcbf-39c8d031aae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6801","nonce":2576774646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6804","nonce":2576774646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6813","nonce":2576774646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6808","nonce":2576774646}]},"public_addr":"172.21.15.64:6801/2576774646","cluster_addr":"172.21.15.64:6804/2576774646","heartbeat_back_addr":"172.21.15.64:6813/2576774646","heartbeat_front_addr":"172.21.15.64:6808/2576774646","state":["exists","up"]},{"osd":7,"uuid":"97e5ef36-58a4-4667-9f9e-b8ced5904d69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6803","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6805","nonce":368918691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6807","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6809","nonce":368918691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6815","nonce":368918691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6811","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6812","nonce":368918691}]},"public_addr":"172.21.15.64:6805/368918691","cluster_addr":"172.21.15.64:6809/368918691","heartbeat_back_addr":"172.21.15.64:6815/368918691","heartbeat_front_addr":"172.21.15.64:6812/368918691","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.862785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.059718+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.575165+0000","dead_epoch":141},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.847666+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.471904+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.061712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.541383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.072272+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.a","osds":[4,1]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T07:22:34.589 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-12T07:22:34.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T07:22:34.891 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:22:34.891 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":186,"fsid":"8412ed58-ac5e-4549-b523-1a58cf9f9429","created":"2024-09-12T07:19:10.328057+0000","modified":"2024-09-12T07:22:34.246048+0000","last_up_change":"2024-09-12T07:22:08.169187+0000","last_in_change":"2024-09-12T07:22:18.153362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T07:19:23.507151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T07:19:28.593632+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-12T07:19:40.555421+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":163,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"710c25c3-92d3-4b51-a7fc-670879faa69c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6817","nonce":4081287959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6819","nonce":4081287959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6823","nonce":4081287959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6821","nonce":4081287959}]},"public_addr":"172.21.15.46:6817/4081287959","cluster_addr":"172.21.15.46:6819/4081287959","heartbeat_back_addr":"172.21.15.46:6823/4081287959","heartbeat_front_addr":"172.21.15.46:6821/4081287959","state":["exists","up"]},{"osd":1,"uuid":"04479191-c52e-4b60-aa0d-3e911f869062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6825","nonce":3980488213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6827","nonce":3980488213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6831","nonce":3980488213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6829","nonce":3980488213}]},"public_addr":"172.21.15.46:6825/3980488213","cluster_addr":"172.21.15.46:6827/3980488213","heartbeat_back_addr":"172.21.15.46:6831/3980488213","heartbeat_front_addr":"172.21.15.46:6829/3980488213","state":["exists","up"]},{"osd":2,"uuid":"ba673e51-96aa-424b-a4df-26432d633ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":140,"up_from":165,"up_thru":172,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3239477423},{"type":"v1","addr":"172.21.15.46:6801","nonce":3239477423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3239477423},{"type":"v1","addr":"172.21.15.46:6803","nonce":3239477423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3239477423},{"type":"v1","addr":"172.21.15.46:6807","nonce":3239477423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3239477423},{"type":"v1","addr":"172.21.15.46:6805","nonce":3239477423}]},"public_addr":"172.21.15.46:6801/3239477423","cluster_addr":"172.21.15.46:6803/3239477423","heartbeat_back_addr":"172.21.15.46:6807/3239477423","heartbeat_front_addr":"172.21.15.46:6805/3239477423","state":["exists","up"]},{"osd":3,"uuid":"c516e59c-834c-4fb0-9ff8-bcc80b21d28a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6809","nonce":3848260327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6811","nonce":3848260327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6815","nonce":3848260327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6813","nonce":3848260327}]},"public_addr":"172.21.15.46:6809/3848260327","cluster_addr":"172.21.15.46:6811/3848260327","heartbeat_back_addr":"172.21.15.46:6815/3848260327","heartbeat_front_addr":"172.21.15.46:6813/3848260327","state":["exists","up"]},{"osd":4,"uuid":"450a70cd-ebb2-48ed-a766-184f58e7032e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6817","nonce":3464256744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6819","nonce":3464256744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6823","nonce":3464256744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6821","nonce":3464256744}]},"public_addr":"172.21.15.64:6817/3464256744","cluster_addr":"172.21.15.64:6819/3464256744","heartbeat_back_addr":"172.21.15.64:6823/3464256744","heartbeat_front_addr":"172.21.15.64:6821/3464256744","state":["exists","up"]},{"osd":5,"uuid":"a4efa333-bf57-45e7-a281-1d3bc9d2e82b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6825","nonce":1789414693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6827","nonce":1789414693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6831","nonce":1789414693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6829","nonce":1789414693}]},"public_addr":"172.21.15.64:6825/1789414693","cluster_addr":"172.21.15.64:6827/1789414693","heartbeat_back_addr":"172.21.15.64:6831/1789414693","heartbeat_front_addr":"172.21.15.64:6829/1789414693","state":["exists","up"]},{"osd":6,"uuid":"bcaad352-f9ee-426c-bcbf-39c8d031aae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6801","nonce":2576774646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6804","nonce":2576774646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6813","nonce":2576774646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6808","nonce":2576774646}]},"public_addr":"172.21.15.64:6801/2576774646","cluster_addr":"172.21.15.64:6804/2576774646","heartbeat_back_addr":"172.21.15.64:6813/2576774646","heartbeat_front_addr":"172.21.15.64:6808/2576774646","state":["exists","up"]},{"osd":7,"uuid":"97e5ef36-58a4-4667-9f9e-b8ced5904d69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6803","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6805","nonce":368918691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6807","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6809","nonce":368918691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6815","nonce":368918691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6811","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6812","nonce":368918691}]},"public_addr":"172.21.15.64:6805/368918691","cluster_addr":"172.21.15.64:6809/368918691","heartbeat_back_addr":"172.21.15.64:6815/368918691","heartbeat_front_addr":"172.21.15.64:6812/368918691","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.862785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.059718+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.575165+0000","dead_epoch":141},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.847666+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.471904+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.061712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.541383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.072272+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.a","osds":[4,1]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T07:22:34.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-12T07:22:34.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-12T07:22:34.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-12T07:22:34.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-12T07:22:34.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-12T07:22:34.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-12T07:22:34.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-12T07:22:34.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-12T07:22:35.158 INFO:teuthology.orchestra.run.smithi046.stdout:708669603847 2024-09-12T07:22:35.159 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T07:22:35.160 INFO:teuthology.orchestra.run.smithi046.stdout:60129542185 2024-09-12T07:22:35.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T07:22:35.163 INFO:teuthology.orchestra.run.smithi046.stdout:60129542186 2024-09-12T07:22:35.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T07:22:35.164 INFO:teuthology.orchestra.run.smithi046.stdout:60129542186 2024-09-12T07:22:35.165 INFO:teuthology.orchestra.run.smithi046.stdout:60129542186 2024-09-12T07:22:35.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T07:22:35.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T07:22:35.168 INFO:teuthology.orchestra.run.smithi046.stdout:60129542187 2024-09-12T07:22:35.168 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T07:22:35.169 INFO:teuthology.orchestra.run.smithi046.stdout:60129542186 2024-09-12T07:22:35.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T07:22:35.170 INFO:teuthology.orchestra.run.smithi046.stdout:60129542186 2024-09-12T07:22:35.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T07:22:35.758 INFO:teuthology.orchestra.run.smithi046.stdout:60129542184 2024-09-12T07:22:35.759 INFO:teuthology.orchestra.run.smithi046.stdout:708669603846 2024-09-12T07:22:35.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 708669603847 got 708669603846 for osd.2 2024-09-12T07:22:35.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542184 for osd.4 2024-09-12T07:22:35.807 INFO:teuthology.orchestra.run.smithi046.stdout:60129542185 2024-09-12T07:22:35.808 INFO:teuthology.orchestra.run.smithi046.stdout:60129542186 2024-09-12T07:22:35.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542185 for osd.5 2024-09-12T07:22:35.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542186 for osd.1 2024-09-12T07:22:35.840 INFO:teuthology.orchestra.run.smithi046.stdout:60129542185 2024-09-12T07:22:35.843 INFO:teuthology.orchestra.run.smithi046.stdout:60129542185 2024-09-12T07:22:35.848 INFO:teuthology.orchestra.run.smithi046.stdout:60129542185 2024-09-12T07:22:35.848 INFO:teuthology.orchestra.run.smithi046.stdout:60129542185 2024-09-12T07:22:35.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542185 for osd.0 2024-09-12T07:22:35.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542185 for osd.7 2024-09-12T07:22:35.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542185 for osd.3 2024-09-12T07:22:35.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542185 for osd.6 2024-09-12T07:22:36.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T07:22:36.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T07:22:36.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T07:22:36.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T07:22:36.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T07:22:36.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T07:22:36.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T07:22:36.867 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T07:22:37.332 INFO:teuthology.orchestra.run.smithi046.stdout:60129542185 2024-09-12T07:22:37.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542185 for osd.4 2024-09-12T07:22:37.353 DEBUG:teuthology.parallel:result is None 2024-09-12T07:22:37.360 INFO:teuthology.orchestra.run.smithi046.stdout:708669603847 2024-09-12T07:22:37.381 INFO:tasks.ceph.ceph_manager.ceph:need seq 708669603847 got 708669603847 for osd.2 2024-09-12T07:22:37.381 DEBUG:teuthology.parallel:result is None 2024-09-12T07:22:37.422 INFO:teuthology.orchestra.run.smithi046.stdout:60129542187 2024-09-12T07:22:37.426 INFO:teuthology.orchestra.run.smithi046.stdout:60129542186 2024-09-12T07:22:37.429 INFO:teuthology.orchestra.run.smithi046.stdout:60129542186 2024-09-12T07:22:37.437 INFO:teuthology.orchestra.run.smithi046.stdout:60129542186 2024-09-12T07:22:37.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542186 for osd.7 2024-09-12T07:22:37.443 DEBUG:teuthology.parallel:result is None 2024-09-12T07:22:37.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542187 for osd.1 2024-09-12T07:22:37.445 DEBUG:teuthology.parallel:result is None 2024-09-12T07:22:37.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542186 for osd.6 2024-09-12T07:22:37.452 DEBUG:teuthology.parallel:result is None 2024-09-12T07:22:37.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542186 for osd.5 2024-09-12T07:22:37.452 DEBUG:teuthology.parallel:result is None 2024-09-12T07:22:37.469 INFO:teuthology.orchestra.run.smithi046.stdout:60129542186 2024-09-12T07:22:37.472 INFO:teuthology.orchestra.run.smithi046.stdout:60129542186 2024-09-12T07:22:37.482 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542186 for osd.0 2024-09-12T07:22:37.482 DEBUG:teuthology.parallel:result is None 2024-09-12T07:22:37.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542186 for osd.3 2024-09-12T07:22:37.485 DEBUG:teuthology.parallel:result is None 2024-09-12T07:22:37.485 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-12T07:22:37.486 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:22:37.831 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:22:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:22:37.843 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-12T07:22:37.525611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":327,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":319,"num_read":260651,"num_read_kb":1191019,"num_write":20586,"num_write_kb":598977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5849,"num_bytes_recovered":432794875,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18034,"ondisk_log_size":18034,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417116,"kb_used_data":18668,"kb_used_omap":34,"kb_used_meta":398237,"kb_avail":754557604,"statfs":{"total":773094113280,"available":772666986496,"internally_reserved":0,"allocated":19116032,"data_stored":9826090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35370,"internal_metadata":407795158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001526"},"pg_stats":[{"pgid":"3.1f","version":"166'638","reported_seq":6785,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264251+0000","last_change":"2024-09-12T07:22:22.149919+0000","last_active":"2024-09-12T07:22:34.264251+0000","last_peered":"2024-09-12T07:22:34.264251+0000","last_clean":"2024-09-12T07:22:34.264251+0000","last_became_active":"2024-09-12T07:22:22.148401+0000","last_became_peered":"2024-09-12T07:22:22.148401+0000","last_unstale":"2024-09-12T07:22:34.264251+0000","last_undegraded":"2024-09-12T07:22:34.264251+0000","last_fullsized":"2024-09-12T07:22:34.264251+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:27:39.529993+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.029437784000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7771,"num_read_kb":29649,"num_write":783,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":2971697,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"161'489","reported_seq":2632,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707535+0000","last_change":"2024-09-12T07:22:06.175231+0000","last_active":"2024-09-12T07:22:34.707535+0000","last_peered":"2024-09-12T07:22:34.707535+0000","last_clean":"2024-09-12T07:22:34.707535+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:22:34.707535+0000","last_undegraded":"2024-09-12T07:22:34.707535+0000","last_fullsized":"2024-09-12T07:22:34.707535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:33:19.436850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026801759999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20338,"num_read_kb":38377,"num_write":424,"num_write_kb":22354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"163'468","reported_seq":21373,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765348+0000","last_change":"2024-09-12T07:22:22.151697+0000","last_active":"2024-09-12T07:22:34.765348+0000","last_peered":"2024-09-12T07:22:34.765348+0000","last_clean":"2024-09-12T07:22:34.765348+0000","last_became_active":"2024-09-12T07:22:22.149491+0000","last_became_peered":"2024-09-12T07:22:22.149491+0000","last_unstale":"2024-09-12T07:22:34.765348+0000","last_undegraded":"2024-09-12T07:22:34.765348+0000","last_fullsized":"2024-09-12T07:22:34.765348+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:51.871543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010058409999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":20924,"num_read_kb":39045,"num_write":507,"num_write_kb":15255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":23048347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"163'607","reported_seq":3893,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764874+0000","last_change":"2024-09-12T07:22:21.150401+0000","last_active":"2024-09-12T07:22:34.764874+0000","last_peered":"2024-09-12T07:22:34.764874+0000","last_clean":"2024-09-12T07:22:34.764874+0000","last_became_active":"2024-09-12T07:22:21.148942+0000","last_became_peered":"2024-09-12T07:22:21.148942+0000","last_unstale":"2024-09-12T07:22:34.764874+0000","last_undegraded":"2024-09-12T07:22:34.764874+0000","last_fullsized":"2024-09-12T07:22:34.764874+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:45.508340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014959909999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":3878,"num_read_kb":17362,"num_write":791,"num_write_kb":11625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":8199265,"num_keys_recovered":174,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"162'447","reported_seq":22008,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264233+0000","last_change":"2024-09-12T07:22:06.195928+0000","last_active":"2024-09-12T07:22:34.264233+0000","last_peered":"2024-09-12T07:22:34.264233+0000","last_clean":"2024-09-12T07:22:34.264233+0000","last_became_active":"2024-09-12T07:22:01.145404+0000","last_became_peered":"2024-09-12T07:22:01.145404+0000","last_unstale":"2024-09-12T07:22:34.264233+0000","last_undegraded":"2024-09-12T07:22:34.264233+0000","last_fullsized":"2024-09-12T07:22:34.264233+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:03.385436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014847230000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":21866,"num_read_kb":61051,"num_write":426,"num_write_kb":15720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":12377120,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'388","reported_seq":4868,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249862+0000","last_change":"2024-09-12T07:22:27.215825+0000","last_active":"2024-09-12T07:22:34.249862+0000","last_peered":"2024-09-12T07:22:34.249862+0000","last_clean":"2024-09-12T07:22:34.249862+0000","last_became_active":"2024-09-12T07:22:27.213519+0000","last_became_peered":"2024-09-12T07:22:27.213519+0000","last_unstale":"2024-09-12T07:22:34.249862+0000","last_undegraded":"2024-09-12T07:22:34.249862+0000","last_fullsized":"2024-09-12T07:22:34.249862+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.924110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022138280000000001,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4657,"num_read_kb":13211,"num_write":331,"num_write_kb":17325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":27687211,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"163'649","reported_seq":7971,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764092+0000","last_change":"2024-09-12T07:22:21.149665+0000","last_active":"2024-09-12T07:22:34.764092+0000","last_peered":"2024-09-12T07:22:34.764092+0000","last_clean":"2024-09-12T07:22:34.764092+0000","last_became_active":"2024-09-12T07:22:21.147583+0000","last_became_peered":"2024-09-12T07:22:21.147583+0000","last_unstale":"2024-09-12T07:22:34.764092+0000","last_undegraded":"2024-09-12T07:22:34.764092+0000","last_fullsized":"2024-09-12T07:22:34.764092+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:22.335622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023936500000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8436,"num_read_kb":50796,"num_write":779,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":7815100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"163'609","reported_seq":3818,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707368+0000","last_change":"2024-09-12T07:22:06.173059+0000","last_active":"2024-09-12T07:22:34.707368+0000","last_peered":"2024-09-12T07:22:34.707368+0000","last_clean":"2024-09-12T07:22:34.707368+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:22:34.707368+0000","last_undegraded":"2024-09-12T07:22:34.707368+0000","last_fullsized":"2024-09-12T07:22:34.707368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:17.727732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099897500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6008,"num_read_kb":52191,"num_write":744,"num_write_kb":22227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"162'653","reported_seq":7128,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264260+0000","last_change":"2024-09-12T07:22:22.149915+0000","last_active":"2024-09-12T07:22:34.264260+0000","last_peered":"2024-09-12T07:22:34.264260+0000","last_clean":"2024-09-12T07:22:34.264260+0000","last_became_active":"2024-09-12T07:22:22.148392+0000","last_became_peered":"2024-09-12T07:22:22.148392+0000","last_unstale":"2024-09-12T07:22:34.264260+0000","last_undegraded":"2024-09-12T07:22:34.264260+0000","last_fullsized":"2024-09-12T07:22:34.264260+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:29.255739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002496505,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8906,"num_read_kb":19832,"num_write":605,"num_write_kb":22990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":20211745,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"163'375","reported_seq":1927,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765128+0000","last_change":"2024-09-12T07:22:21.149525+0000","last_active":"2024-09-12T07:22:34.765128+0000","last_peered":"2024-09-12T07:22:34.765128+0000","last_clean":"2024-09-12T07:22:34.765128+0000","last_became_active":"2024-09-12T07:22:21.148026+0000","last_became_peered":"2024-09-12T07:22:21.148026+0000","last_unstale":"2024-09-12T07:22:34.765128+0000","last_undegraded":"2024-09-12T07:22:34.765128+0000","last_fullsized":"2024-09-12T07:22:34.765128+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:15.887884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017138209999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1348,"num_read_kb":18539,"num_write":421,"num_write_kb":14413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":10121363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"152'344","reported_seq":7171,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765014+0000","last_change":"2024-09-12T07:22:06.164819+0000","last_active":"2024-09-12T07:22:34.765014+0000","last_peered":"2024-09-12T07:22:34.765014+0000","last_clean":"2024-09-12T07:22:34.765014+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:22:34.765014+0000","last_undegraded":"2024-09-12T07:22:34.765014+0000","last_fullsized":"2024-09-12T07:22:34.765014+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:45.370726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080767000000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7070,"num_read_kb":53310,"num_write":299,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"160'391","reported_seq":5850,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.250009+0000","last_change":"2024-09-12T07:22:27.214683+0000","last_active":"2024-09-12T07:22:34.250009+0000","last_peered":"2024-09-12T07:22:34.250009+0000","last_clean":"2024-09-12T07:22:34.250009+0000","last_became_active":"2024-09-12T07:22:27.212420+0000","last_became_peered":"2024-09-12T07:22:27.212420+0000","last_unstale":"2024-09-12T07:22:34.250009+0000","last_undegraded":"2024-09-12T07:22:34.250009+0000","last_fullsized":"2024-09-12T07:22:34.250009+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:16.234305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097451600000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5380,"num_read_kb":60804,"num_write":390,"num_write_kb":30252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":31149476,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764999+0000","last_change":"2024-09-12T07:22:31.308130+0000","last_active":"2024-09-12T07:22:34.764999+0000","last_peered":"2024-09-12T07:22:34.764999+0000","last_clean":"2024-09-12T07:22:34.764999+0000","last_became_active":"2024-09-12T07:22:31.307821+0000","last_became_peered":"2024-09-12T07:22:31.307821+0000","last_unstale":"2024-09-12T07:22:34.764999+0000","last_undegraded":"2024-09-12T07:22:34.764999+0000","last_fullsized":"2024-09-12T07:22:34.764999+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:45.276396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"163'639","reported_seq":6266,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264231+0000","last_change":"2024-09-12T07:22:21.153348+0000","last_active":"2024-09-12T07:22:34.264231+0000","last_peered":"2024-09-12T07:22:34.264231+0000","last_clean":"2024-09-12T07:22:34.264231+0000","last_became_active":"2024-09-12T07:22:21.152036+0000","last_became_peered":"2024-09-12T07:22:21.152036+0000","last_unstale":"2024-09-12T07:22:34.264231+0000","last_undegraded":"2024-09-12T07:22:34.264231+0000","last_fullsized":"2024-09-12T07:22:34.264231+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:16.901686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014037019999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6009,"num_read_kb":23012,"num_write":704,"num_write_kb":11415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":1480654,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764229+0000","last_change":"2024-09-12T07:22:18.173512+0000","last_active":"2024-09-12T07:22:34.764229+0000","last_peered":"2024-09-12T07:22:34.764229+0000","last_clean":"2024-09-12T07:22:34.764229+0000","last_became_active":"2024-09-12T07:22:18.173148+0000","last_became_peered":"2024-09-12T07:22:18.173148+0000","last_unstale":"2024-09-12T07:22:34.764229+0000","last_undegraded":"2024-09-12T07:22:34.764229+0000","last_fullsized":"2024-09-12T07:22:34.764229+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:07.135570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"162'679","reported_seq":2209,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707544+0000","last_change":"2024-09-12T07:22:06.174857+0000","last_active":"2024-09-12T07:22:34.707544+0000","last_peered":"2024-09-12T07:22:34.707544+0000","last_clean":"2024-09-12T07:22:34.707544+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:22:34.707544+0000","last_undegraded":"2024-09-12T07:22:34.707544+0000","last_fullsized":"2024-09-12T07:22:34.707544+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:16.561231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025712959999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3057,"num_read_kb":14289,"num_write":898,"num_write_kb":11329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":348,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764698+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:22:34.764698+0000","last_peered":"2024-09-12T07:22:34.764698+0000","last_clean":"2024-09-12T07:22:34.764698+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:22:34.764698+0000","last_undegraded":"2024-09-12T07:22:34.764698+0000","last_fullsized":"2024-09-12T07:22:34.764698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"166'506","reported_seq":3850,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764186+0000","last_change":"2024-09-12T07:22:21.149345+0000","last_active":"2024-09-12T07:22:34.764186+0000","last_peered":"2024-09-12T07:22:34.764186+0000","last_clean":"2024-09-12T07:22:34.764186+0000","last_became_active":"2024-09-12T07:22:21.147703+0000","last_became_peered":"2024-09-12T07:22:21.147703+0000","last_unstale":"2024-09-12T07:22:34.764186+0000","last_undegraded":"2024-09-12T07:22:34.764186+0000","last_fullsized":"2024-09-12T07:22:34.764186+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:05.437283+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.023667932999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4682,"num_read_kb":14844,"num_write":579,"num_write_kb":8813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6163290,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764154+0000","last_change":"2024-09-12T07:22:18.169631+0000","last_active":"2024-09-12T07:22:34.764154+0000","last_peered":"2024-09-12T07:22:34.764154+0000","last_clean":"2024-09-12T07:22:34.764154+0000","last_became_active":"2024-09-12T07:22:18.169285+0000","last_became_peered":"2024-09-12T07:22:18.169285+0000","last_unstale":"2024-09-12T07:22:34.764154+0000","last_undegraded":"2024-09-12T07:22:34.764154+0000","last_fullsized":"2024-09-12T07:22:34.764154+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:21.373190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"166'565","reported_seq":10840,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765459+0000","last_change":"2024-09-12T07:22:22.153784+0000","last_active":"2024-09-12T07:22:34.765459+0000","last_peered":"2024-09-12T07:22:34.765459+0000","last_clean":"2024-09-12T07:22:34.765459+0000","last_became_active":"2024-09-12T07:22:22.151818+0000","last_became_peered":"2024-09-12T07:22:22.151818+0000","last_unstale":"2024-09-12T07:22:34.765459+0000","last_undegraded":"2024-09-12T07:22:34.765459+0000","last_fullsized":"2024-09-12T07:22:34.765459+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:02.130727+0000","scrub_duration":0,"objects_trimmed":18,"snaptrim_duration":0.023276249999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11624,"num_read_kb":77562,"num_write":595,"num_write_kb":25411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":22089334,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"163'503","reported_seq":4495,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249801+0000","last_change":"2024-09-12T07:22:27.215301+0000","last_active":"2024-09-12T07:22:34.249801+0000","last_peered":"2024-09-12T07:22:34.249801+0000","last_clean":"2024-09-12T07:22:34.249801+0000","last_became_active":"2024-09-12T07:22:27.213073+0000","last_became_peered":"2024-09-12T07:22:27.213073+0000","last_unstale":"2024-09-12T07:22:34.249801+0000","last_undegraded":"2024-09-12T07:22:34.249801+0000","last_fullsized":"2024-09-12T07:22:34.249801+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:10.316262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018780750000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":3159,"num_read_kb":12201,"num_write":460,"num_write_kb":10579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":2603172,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":48,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765232+0000","last_change":"2024-09-12T07:22:19.200232+0000","last_active":"2024-09-12T07:22:34.765232+0000","last_peered":"2024-09-12T07:22:34.765232+0000","last_clean":"2024-09-12T07:22:34.765232+0000","last_became_active":"2024-09-12T07:22:19.178595+0000","last_became_peered":"2024-09-12T07:22:19.178595+0000","last_unstale":"2024-09-12T07:22:34.765232+0000","last_undegraded":"2024-09-12T07:22:34.765232+0000","last_fullsized":"2024-09-12T07:22:34.765232+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:58.776762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"166'781","reported_seq":3325,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764213+0000","last_change":"2024-09-12T07:22:21.150766+0000","last_active":"2024-09-12T07:22:34.764213+0000","last_peered":"2024-09-12T07:22:34.764213+0000","last_clean":"2024-09-12T07:22:34.764213+0000","last_became_active":"2024-09-12T07:22:21.147758+0000","last_became_peered":"2024-09-12T07:22:21.147758+0000","last_unstale":"2024-09-12T07:22:34.764213+0000","last_undegraded":"2024-09-12T07:22:34.764213+0000","last_fullsized":"2024-09-12T07:22:34.764213+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:15.197467+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.033858052999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5450,"num_read_kb":11647,"num_write":1052,"num_write_kb":13386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":9483314,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":299,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249699+0000","last_change":"2024-09-12T07:22:09.183569+0000","last_active":"2024-09-12T07:22:34.249699+0000","last_peered":"2024-09-12T07:22:34.249699+0000","last_clean":"2024-09-12T07:22:34.249699+0000","last_became_active":"2024-09-12T07:22:09.182547+0000","last_became_peered":"2024-09-12T07:22:09.182547+0000","last_unstale":"2024-09-12T07:22:34.249699+0000","last_undegraded":"2024-09-12T07:22:34.249699+0000","last_fullsized":"2024-09-12T07:22:34.249699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:52.520157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"162'406","reported_seq":10545,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264069+0000","last_change":"2024-09-12T07:22:06.195337+0000","last_active":"2024-09-12T07:22:34.264069+0000","last_peered":"2024-09-12T07:22:34.264069+0000","last_clean":"2024-09-12T07:22:34.264069+0000","last_became_active":"2024-09-12T07:22:01.146421+0000","last_became_peered":"2024-09-12T07:22:01.146421+0000","last_unstale":"2024-09-12T07:22:34.264069+0000","last_undegraded":"2024-09-12T07:22:34.264069+0000","last_fullsized":"2024-09-12T07:22:34.264069+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:38.812988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094442799999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11174,"num_read_kb":59795,"num_write":376,"num_write_kb":27716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":22543830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":360,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764760+0000","last_change":"2024-09-12T07:22:18.171696+0000","last_active":"2024-09-12T07:22:34.764760+0000","last_peered":"2024-09-12T07:22:34.764760+0000","last_clean":"2024-09-12T07:22:34.764760+0000","last_became_active":"2024-09-12T07:22:18.171266+0000","last_became_peered":"2024-09-12T07:22:18.171266+0000","last_unstale":"2024-09-12T07:22:34.764760+0000","last_undegraded":"2024-09-12T07:22:34.764760+0000","last_fullsized":"2024-09-12T07:22:34.764760+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:26.361507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"162'1062","reported_seq":6945,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707508+0000","last_change":"2024-09-12T07:22:22.152024+0000","last_active":"2024-09-12T07:22:34.707508+0000","last_peered":"2024-09-12T07:22:34.707508+0000","last_clean":"2024-09-12T07:22:34.707508+0000","last_became_active":"2024-09-12T07:22:22.150316+0000","last_became_peered":"2024-09-12T07:22:22.150316+0000","last_unstale":"2024-09-12T07:22:34.707508+0000","last_undegraded":"2024-09-12T07:22:34.707508+0000","last_fullsized":"2024-09-12T07:22:34.707508+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:57:13.384646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021189680000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8933,"num_read_kb":59814,"num_write":1570,"num_write_kb":27462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":14379761,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":45,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764860+0000","last_change":"2024-09-12T07:22:19.178038+0000","last_active":"2024-09-12T07:22:34.764860+0000","last_peered":"2024-09-12T07:22:34.764860+0000","last_clean":"2024-09-12T07:22:34.764860+0000","last_became_active":"2024-09-12T07:22:19.170715+0000","last_became_peered":"2024-09-12T07:22:19.170715+0000","last_unstale":"2024-09-12T07:22:34.764860+0000","last_undegraded":"2024-09-12T07:22:34.764860+0000","last_fullsized":"2024-09-12T07:22:34.764860+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:28.982893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":439,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764900+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:22:34.764900+0000","last_peered":"2024-09-12T07:22:34.764900+0000","last_clean":"2024-09-12T07:22:34.764900+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:22:34.764900+0000","last_undegraded":"2024-09-12T07:22:34.764900+0000","last_fullsized":"2024-09-12T07:22:34.764900+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"162'962","reported_seq":3107,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707372+0000","last_change":"2024-09-12T07:22:21.151517+0000","last_active":"2024-09-12T07:22:34.707372+0000","last_peered":"2024-09-12T07:22:34.707372+0000","last_clean":"2024-09-12T07:22:34.707372+0000","last_became_active":"2024-09-12T07:22:21.150083+0000","last_became_peered":"2024-09-12T07:22:21.150083+0000","last_unstale":"2024-09-12T07:22:34.707372+0000","last_undegraded":"2024-09-12T07:22:34.707372+0000","last_fullsized":"2024-09-12T07:22:34.707372+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":962,"log_dups_size":0,"ondisk_log_size":962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:38.471826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011038020000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5044,"num_read_kb":23221,"num_write":1448,"num_write_kb":24489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":21462395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"163'549","reported_seq":5426,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249824+0000","last_change":"2024-09-12T07:22:27.215048+0000","last_active":"2024-09-12T07:22:34.249824+0000","last_peered":"2024-09-12T07:22:34.249824+0000","last_clean":"2024-09-12T07:22:34.249824+0000","last_became_active":"2024-09-12T07:22:27.212942+0000","last_became_peered":"2024-09-12T07:22:27.212942+0000","last_unstale":"2024-09-12T07:22:34.249824+0000","last_undegraded":"2024-09-12T07:22:34.249824+0000","last_fullsized":"2024-09-12T07:22:34.249824+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:59.386875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021244950000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7563,"num_read_kb":32357,"num_write":525,"num_write_kb":20455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1710555,"num_keys_recovered":41,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"162'540","reported_seq":6677,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249766+0000","last_change":"2024-09-12T07:22:27.215838+0000","last_active":"2024-09-12T07:22:34.249766+0000","last_peered":"2024-09-12T07:22:34.249766+0000","last_clean":"2024-09-12T07:22:34.249766+0000","last_became_active":"2024-09-12T07:22:27.213533+0000","last_became_peered":"2024-09-12T07:22:27.213533+0000","last_unstale":"2024-09-12T07:22:34.249766+0000","last_undegraded":"2024-09-12T07:22:34.249766+0000","last_fullsized":"2024-09-12T07:22:34.249766+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:38.800561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069883599999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":6247,"num_read_kb":49000,"num_write":522,"num_write_kb":25535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":42565291,"num_keys_recovered":104,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"162'692","reported_seq":3567,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707411+0000","last_change":"2024-09-12T07:22:06.173724+0000","last_active":"2024-09-12T07:22:34.707411+0000","last_peered":"2024-09-12T07:22:34.707411+0000","last_clean":"2024-09-12T07:22:34.707411+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:22:34.707411+0000","last_undegraded":"2024-09-12T07:22:34.707411+0000","last_fullsized":"2024-09-12T07:22:34.707411+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:30.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015261610000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6009,"num_read_kb":21069,"num_write":727,"num_write_kb":17071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"162'445","reported_seq":9922,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764933+0000","last_change":"2024-09-12T07:22:22.151400+0000","last_active":"2024-09-12T07:22:34.764933+0000","last_peered":"2024-09-12T07:22:34.764933+0000","last_clean":"2024-09-12T07:22:34.764933+0000","last_became_active":"2024-09-12T07:22:22.149715+0000","last_became_peered":"2024-09-12T07:22:22.149715+0000","last_unstale":"2024-09-12T07:22:34.764933+0000","last_undegraded":"2024-09-12T07:22:34.764933+0000","last_fullsized":"2024-09-12T07:22:34.764933+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:51.662782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016506240000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11019,"num_read_kb":49685,"num_write":382,"num_write_kb":15924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":18973449,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"161'769","reported_seq":3640,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765503+0000","last_change":"2024-09-12T07:22:22.153946+0000","last_active":"2024-09-12T07:22:34.765503+0000","last_peered":"2024-09-12T07:22:34.765503+0000","last_clean":"2024-09-12T07:22:34.765503+0000","last_became_active":"2024-09-12T07:22:22.152455+0000","last_became_peered":"2024-09-12T07:22:22.152455+0000","last_unstale":"2024-09-12T07:22:34.765503+0000","last_undegraded":"2024-09-12T07:22:34.765503+0000","last_fullsized":"2024-09-12T07:22:34.765503+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:17.209337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014515540000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5118,"num_read_kb":30709,"num_write":1176,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":7370075,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"162'531","reported_seq":5409,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765193+0000","last_change":"2024-09-12T07:22:06.166758+0000","last_active":"2024-09-12T07:22:34.765193+0000","last_peered":"2024-09-12T07:22:34.765193+0000","last_clean":"2024-09-12T07:22:34.765193+0000","last_became_active":"2024-09-12T07:22:00.145108+0000","last_became_peered":"2024-09-12T07:22:00.145108+0000","last_unstale":"2024-09-12T07:22:34.765193+0000","last_undegraded":"2024-09-12T07:22:34.765193+0000","last_fullsized":"2024-09-12T07:22:34.765193+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:31.569737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801299999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6080,"num_read_kb":38867,"num_write":581,"num_write_kb":16891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":10508628,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"162'596","reported_seq":3023,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765441+0000","last_change":"2024-09-12T07:22:06.166247+0000","last_active":"2024-09-12T07:22:34.765441+0000","last_peered":"2024-09-12T07:22:34.765441+0000","last_clean":"2024-09-12T07:22:34.765441+0000","last_became_active":"2024-09-12T07:22:01.143359+0000","last_became_peered":"2024-09-12T07:22:01.143359+0000","last_unstale":"2024-09-12T07:22:34.765441+0000","last_undegraded":"2024-09-12T07:22:34.765441+0000","last_fullsized":"2024-09-12T07:22:34.765441+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:49.338904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001515553,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3463,"num_read_kb":13505,"num_write":637,"num_write_kb":16522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":8725508,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"152'338","reported_seq":1797,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765144+0000","last_change":"2024-09-12T07:22:31.247099+0000","last_active":"2024-09-12T07:22:34.765144+0000","last_peered":"2024-09-12T07:22:34.765144+0000","last_clean":"2024-09-12T07:22:34.765144+0000","last_became_active":"2024-09-12T07:22:31.245905+0000","last_became_peered":"2024-09-12T07:22:31.245905+0000","last_unstale":"2024-09-12T07:22:34.765144+0000","last_undegraded":"2024-09-12T07:22:34.765144+0000","last_fullsized":"2024-09-12T07:22:34.765144+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:10.350713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816669999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1472,"num_read_kb":27101,"num_write":327,"num_write_kb":18963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"163'454","reported_seq":7380,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764987+0000","last_change":"2024-09-12T07:22:06.164601+0000","last_active":"2024-09-12T07:22:34.764987+0000","last_peered":"2024-09-12T07:22:34.764987+0000","last_clean":"2024-09-12T07:22:34.764987+0000","last_became_active":"2024-09-12T07:22:01.143631+0000","last_became_peered":"2024-09-12T07:22:01.143631+0000","last_unstale":"2024-09-12T07:22:34.764987+0000","last_undegraded":"2024-09-12T07:22:34.764987+0000","last_fullsized":"2024-09-12T07:22:34.764987+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:34.629049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065451099999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7090,"num_read_kb":47369,"num_write":461,"num_write_kb":21538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":17766842,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"159'408","reported_seq":11210,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264074+0000","last_change":"2024-09-12T07:22:06.196502+0000","last_active":"2024-09-12T07:22:34.264074+0000","last_peered":"2024-09-12T07:22:34.264074+0000","last_clean":"2024-09-12T07:22:34.264074+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:22:34.264074+0000","last_undegraded":"2024-09-12T07:22:34.264074+0000","last_fullsized":"2024-09-12T07:22:34.264074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:48.940535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019415999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11863,"num_read_kb":47954,"num_write":410,"num_write_kb":15586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'476","reported_seq":19728,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765222+0000","last_change":"2024-09-12T07:22:06.165173+0000","last_active":"2024-09-12T07:22:34.765222+0000","last_peered":"2024-09-12T07:22:34.765222+0000","last_clean":"2024-09-12T07:22:34.765222+0000","last_became_active":"2024-09-12T07:22:01.143577+0000","last_became_peered":"2024-09-12T07:22:01.143577+0000","last_unstale":"2024-09-12T07:22:34.765222+0000","last_undegraded":"2024-09-12T07:22:34.765222+0000","last_fullsized":"2024-09-12T07:22:34.765222+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:12:53.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010515870000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":18947,"num_read_kb":82796,"num_write":553,"num_write_kb":44274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":53090062,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":323,"num_object_clones":0,"num_object_copies":646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":319,"num_read":260585,"num_read_kb":1190964,"num_write":20483,"num_write_kb":597347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5845,"num_bytes_recovered":432794818,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},"log_size":17959,"ondisk_log_size":17959,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56164,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315676,"statfs":{"total":96636764160,"available":96579252224,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33376,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338464,"statfs":{"total":96636764160,"available":96602587136,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55576,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316264,"statfs":{"total":96636764160,"available":96579854336,"internally_reserved":0,"allocated":2232320,"data_stored":1075567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51432,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320408,"statfs":{"total":96636764160,"available":96584097792,"internally_reserved":0,"allocated":2244608,"data_stored":1081409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69480,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302360,"statfs":{"total":96636764160,"available":96565616640,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":165,"seq":708669603848,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31008,"kb_used_data":2188,"kb_used_omap":23,"kb_used_meta":28776,"kb_avail":94340832,"statfs":{"total":96636764160,"available":96605011968,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24213,"internal_metadata":29466987},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55592,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316248,"statfs":{"total":96636764160,"available":96579837952,"internally_reserved":0,"allocated":2248704,"data_stored":1081428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64488,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307352,"statfs":{"total":96636764160,"available":96570728448,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:22:37.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:22:38.144 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:22:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:22:38.157 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-12T07:22:37.525611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":327,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":319,"num_read":260651,"num_read_kb":1191019,"num_write":20586,"num_write_kb":598977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5849,"num_bytes_recovered":432794875,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18034,"ondisk_log_size":18034,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417116,"kb_used_data":18668,"kb_used_omap":34,"kb_used_meta":398237,"kb_avail":754557604,"statfs":{"total":773094113280,"available":772666986496,"internally_reserved":0,"allocated":19116032,"data_stored":9826090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35370,"internal_metadata":407795158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001526"},"pg_stats":[{"pgid":"3.1f","version":"166'638","reported_seq":6785,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264251+0000","last_change":"2024-09-12T07:22:22.149919+0000","last_active":"2024-09-12T07:22:34.264251+0000","last_peered":"2024-09-12T07:22:34.264251+0000","last_clean":"2024-09-12T07:22:34.264251+0000","last_became_active":"2024-09-12T07:22:22.148401+0000","last_became_peered":"2024-09-12T07:22:22.148401+0000","last_unstale":"2024-09-12T07:22:34.264251+0000","last_undegraded":"2024-09-12T07:22:34.264251+0000","last_fullsized":"2024-09-12T07:22:34.264251+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:27:39.529993+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.029437784000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7771,"num_read_kb":29649,"num_write":783,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":2971697,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"161'489","reported_seq":2632,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707535+0000","last_change":"2024-09-12T07:22:06.175231+0000","last_active":"2024-09-12T07:22:34.707535+0000","last_peered":"2024-09-12T07:22:34.707535+0000","last_clean":"2024-09-12T07:22:34.707535+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:22:34.707535+0000","last_undegraded":"2024-09-12T07:22:34.707535+0000","last_fullsized":"2024-09-12T07:22:34.707535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:33:19.436850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026801759999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20338,"num_read_kb":38377,"num_write":424,"num_write_kb":22354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"163'468","reported_seq":21373,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765348+0000","last_change":"2024-09-12T07:22:22.151697+0000","last_active":"2024-09-12T07:22:34.765348+0000","last_peered":"2024-09-12T07:22:34.765348+0000","last_clean":"2024-09-12T07:22:34.765348+0000","last_became_active":"2024-09-12T07:22:22.149491+0000","last_became_peered":"2024-09-12T07:22:22.149491+0000","last_unstale":"2024-09-12T07:22:34.765348+0000","last_undegraded":"2024-09-12T07:22:34.765348+0000","last_fullsized":"2024-09-12T07:22:34.765348+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:51.871543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010058409999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":20924,"num_read_kb":39045,"num_write":507,"num_write_kb":15255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":23048347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"163'607","reported_seq":3893,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764874+0000","last_change":"2024-09-12T07:22:21.150401+0000","last_active":"2024-09-12T07:22:34.764874+0000","last_peered":"2024-09-12T07:22:34.764874+0000","last_clean":"2024-09-12T07:22:34.764874+0000","last_became_active":"2024-09-12T07:22:21.148942+0000","last_became_peered":"2024-09-12T07:22:21.148942+0000","last_unstale":"2024-09-12T07:22:34.764874+0000","last_undegraded":"2024-09-12T07:22:34.764874+0000","last_fullsized":"2024-09-12T07:22:34.764874+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:45.508340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014959909999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":3878,"num_read_kb":17362,"num_write":791,"num_write_kb":11625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":8199265,"num_keys_recovered":174,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"162'447","reported_seq":22008,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264233+0000","last_change":"2024-09-12T07:22:06.195928+0000","last_active":"2024-09-12T07:22:34.264233+0000","last_peered":"2024-09-12T07:22:34.264233+0000","last_clean":"2024-09-12T07:22:34.264233+0000","last_became_active":"2024-09-12T07:22:01.145404+0000","last_became_peered":"2024-09-12T07:22:01.145404+0000","last_unstale":"2024-09-12T07:22:34.264233+0000","last_undegraded":"2024-09-12T07:22:34.264233+0000","last_fullsized":"2024-09-12T07:22:34.264233+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:03.385436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014847230000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":21866,"num_read_kb":61051,"num_write":426,"num_write_kb":15720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":12377120,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'388","reported_seq":4868,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249862+0000","last_change":"2024-09-12T07:22:27.215825+0000","last_active":"2024-09-12T07:22:34.249862+0000","last_peered":"2024-09-12T07:22:34.249862+0000","last_clean":"2024-09-12T07:22:34.249862+0000","last_became_active":"2024-09-12T07:22:27.213519+0000","last_became_peered":"2024-09-12T07:22:27.213519+0000","last_unstale":"2024-09-12T07:22:34.249862+0000","last_undegraded":"2024-09-12T07:22:34.249862+0000","last_fullsized":"2024-09-12T07:22:34.249862+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.924110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022138280000000001,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4657,"num_read_kb":13211,"num_write":331,"num_write_kb":17325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":27687211,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"163'649","reported_seq":7971,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764092+0000","last_change":"2024-09-12T07:22:21.149665+0000","last_active":"2024-09-12T07:22:34.764092+0000","last_peered":"2024-09-12T07:22:34.764092+0000","last_clean":"2024-09-12T07:22:34.764092+0000","last_became_active":"2024-09-12T07:22:21.147583+0000","last_became_peered":"2024-09-12T07:22:21.147583+0000","last_unstale":"2024-09-12T07:22:34.764092+0000","last_undegraded":"2024-09-12T07:22:34.764092+0000","last_fullsized":"2024-09-12T07:22:34.764092+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:22.335622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023936500000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8436,"num_read_kb":50796,"num_write":779,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":7815100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"163'609","reported_seq":3818,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707368+0000","last_change":"2024-09-12T07:22:06.173059+0000","last_active":"2024-09-12T07:22:34.707368+0000","last_peered":"2024-09-12T07:22:34.707368+0000","last_clean":"2024-09-12T07:22:34.707368+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:22:34.707368+0000","last_undegraded":"2024-09-12T07:22:34.707368+0000","last_fullsized":"2024-09-12T07:22:34.707368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:17.727732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099897500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6008,"num_read_kb":52191,"num_write":744,"num_write_kb":22227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"162'653","reported_seq":7128,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264260+0000","last_change":"2024-09-12T07:22:22.149915+0000","last_active":"2024-09-12T07:22:34.264260+0000","last_peered":"2024-09-12T07:22:34.264260+0000","last_clean":"2024-09-12T07:22:34.264260+0000","last_became_active":"2024-09-12T07:22:22.148392+0000","last_became_peered":"2024-09-12T07:22:22.148392+0000","last_unstale":"2024-09-12T07:22:34.264260+0000","last_undegraded":"2024-09-12T07:22:34.264260+0000","last_fullsized":"2024-09-12T07:22:34.264260+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:29.255739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002496505,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8906,"num_read_kb":19832,"num_write":605,"num_write_kb":22990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":20211745,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"163'375","reported_seq":1927,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765128+0000","last_change":"2024-09-12T07:22:21.149525+0000","last_active":"2024-09-12T07:22:34.765128+0000","last_peered":"2024-09-12T07:22:34.765128+0000","last_clean":"2024-09-12T07:22:34.765128+0000","last_became_active":"2024-09-12T07:22:21.148026+0000","last_became_peered":"2024-09-12T07:22:21.148026+0000","last_unstale":"2024-09-12T07:22:34.765128+0000","last_undegraded":"2024-09-12T07:22:34.765128+0000","last_fullsized":"2024-09-12T07:22:34.765128+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:15.887884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017138209999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1348,"num_read_kb":18539,"num_write":421,"num_write_kb":14413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":10121363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"152'344","reported_seq":7171,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765014+0000","last_change":"2024-09-12T07:22:06.164819+0000","last_active":"2024-09-12T07:22:34.765014+0000","last_peered":"2024-09-12T07:22:34.765014+0000","last_clean":"2024-09-12T07:22:34.765014+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:22:34.765014+0000","last_undegraded":"2024-09-12T07:22:34.765014+0000","last_fullsized":"2024-09-12T07:22:34.765014+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:45.370726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080767000000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7070,"num_read_kb":53310,"num_write":299,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"160'391","reported_seq":5850,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.250009+0000","last_change":"2024-09-12T07:22:27.214683+0000","last_active":"2024-09-12T07:22:34.250009+0000","last_peered":"2024-09-12T07:22:34.250009+0000","last_clean":"2024-09-12T07:22:34.250009+0000","last_became_active":"2024-09-12T07:22:27.212420+0000","last_became_peered":"2024-09-12T07:22:27.212420+0000","last_unstale":"2024-09-12T07:22:34.250009+0000","last_undegraded":"2024-09-12T07:22:34.250009+0000","last_fullsized":"2024-09-12T07:22:34.250009+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:16.234305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097451600000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5380,"num_read_kb":60804,"num_write":390,"num_write_kb":30252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":31149476,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764999+0000","last_change":"2024-09-12T07:22:31.308130+0000","last_active":"2024-09-12T07:22:34.764999+0000","last_peered":"2024-09-12T07:22:34.764999+0000","last_clean":"2024-09-12T07:22:34.764999+0000","last_became_active":"2024-09-12T07:22:31.307821+0000","last_became_peered":"2024-09-12T07:22:31.307821+0000","last_unstale":"2024-09-12T07:22:34.764999+0000","last_undegraded":"2024-09-12T07:22:34.764999+0000","last_fullsized":"2024-09-12T07:22:34.764999+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:45.276396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"163'639","reported_seq":6266,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264231+0000","last_change":"2024-09-12T07:22:21.153348+0000","last_active":"2024-09-12T07:22:34.264231+0000","last_peered":"2024-09-12T07:22:34.264231+0000","last_clean":"2024-09-12T07:22:34.264231+0000","last_became_active":"2024-09-12T07:22:21.152036+0000","last_became_peered":"2024-09-12T07:22:21.152036+0000","last_unstale":"2024-09-12T07:22:34.264231+0000","last_undegraded":"2024-09-12T07:22:34.264231+0000","last_fullsized":"2024-09-12T07:22:34.264231+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:16.901686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014037019999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6009,"num_read_kb":23012,"num_write":704,"num_write_kb":11415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":1480654,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764229+0000","last_change":"2024-09-12T07:22:18.173512+0000","last_active":"2024-09-12T07:22:34.764229+0000","last_peered":"2024-09-12T07:22:34.764229+0000","last_clean":"2024-09-12T07:22:34.764229+0000","last_became_active":"2024-09-12T07:22:18.173148+0000","last_became_peered":"2024-09-12T07:22:18.173148+0000","last_unstale":"2024-09-12T07:22:34.764229+0000","last_undegraded":"2024-09-12T07:22:34.764229+0000","last_fullsized":"2024-09-12T07:22:34.764229+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:07.135570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"162'679","reported_seq":2209,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707544+0000","last_change":"2024-09-12T07:22:06.174857+0000","last_active":"2024-09-12T07:22:34.707544+0000","last_peered":"2024-09-12T07:22:34.707544+0000","last_clean":"2024-09-12T07:22:34.707544+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:22:34.707544+0000","last_undegraded":"2024-09-12T07:22:34.707544+0000","last_fullsized":"2024-09-12T07:22:34.707544+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:16.561231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025712959999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3057,"num_read_kb":14289,"num_write":898,"num_write_kb":11329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":348,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764698+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:22:34.764698+0000","last_peered":"2024-09-12T07:22:34.764698+0000","last_clean":"2024-09-12T07:22:34.764698+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:22:34.764698+0000","last_undegraded":"2024-09-12T07:22:34.764698+0000","last_fullsized":"2024-09-12T07:22:34.764698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"166'506","reported_seq":3850,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764186+0000","last_change":"2024-09-12T07:22:21.149345+0000","last_active":"2024-09-12T07:22:34.764186+0000","last_peered":"2024-09-12T07:22:34.764186+0000","last_clean":"2024-09-12T07:22:34.764186+0000","last_became_active":"2024-09-12T07:22:21.147703+0000","last_became_peered":"2024-09-12T07:22:21.147703+0000","last_unstale":"2024-09-12T07:22:34.764186+0000","last_undegraded":"2024-09-12T07:22:34.764186+0000","last_fullsized":"2024-09-12T07:22:34.764186+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:05.437283+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.023667932999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4682,"num_read_kb":14844,"num_write":579,"num_write_kb":8813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6163290,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764154+0000","last_change":"2024-09-12T07:22:18.169631+0000","last_active":"2024-09-12T07:22:34.764154+0000","last_peered":"2024-09-12T07:22:34.764154+0000","last_clean":"2024-09-12T07:22:34.764154+0000","last_became_active":"2024-09-12T07:22:18.169285+0000","last_became_peered":"2024-09-12T07:22:18.169285+0000","last_unstale":"2024-09-12T07:22:34.764154+0000","last_undegraded":"2024-09-12T07:22:34.764154+0000","last_fullsized":"2024-09-12T07:22:34.764154+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:21.373190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"166'565","reported_seq":10840,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765459+0000","last_change":"2024-09-12T07:22:22.153784+0000","last_active":"2024-09-12T07:22:34.765459+0000","last_peered":"2024-09-12T07:22:34.765459+0000","last_clean":"2024-09-12T07:22:34.765459+0000","last_became_active":"2024-09-12T07:22:22.151818+0000","last_became_peered":"2024-09-12T07:22:22.151818+0000","last_unstale":"2024-09-12T07:22:34.765459+0000","last_undegraded":"2024-09-12T07:22:34.765459+0000","last_fullsized":"2024-09-12T07:22:34.765459+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:02.130727+0000","scrub_duration":0,"objects_trimmed":18,"snaptrim_duration":0.023276249999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11624,"num_read_kb":77562,"num_write":595,"num_write_kb":25411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":22089334,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"163'503","reported_seq":4495,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249801+0000","last_change":"2024-09-12T07:22:27.215301+0000","last_active":"2024-09-12T07:22:34.249801+0000","last_peered":"2024-09-12T07:22:34.249801+0000","last_clean":"2024-09-12T07:22:34.249801+0000","last_became_active":"2024-09-12T07:22:27.213073+0000","last_became_peered":"2024-09-12T07:22:27.213073+0000","last_unstale":"2024-09-12T07:22:34.249801+0000","last_undegraded":"2024-09-12T07:22:34.249801+0000","last_fullsized":"2024-09-12T07:22:34.249801+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:10.316262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018780750000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":3159,"num_read_kb":12201,"num_write":460,"num_write_kb":10579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":2603172,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":48,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765232+0000","last_change":"2024-09-12T07:22:19.200232+0000","last_active":"2024-09-12T07:22:34.765232+0000","last_peered":"2024-09-12T07:22:34.765232+0000","last_clean":"2024-09-12T07:22:34.765232+0000","last_became_active":"2024-09-12T07:22:19.178595+0000","last_became_peered":"2024-09-12T07:22:19.178595+0000","last_unstale":"2024-09-12T07:22:34.765232+0000","last_undegraded":"2024-09-12T07:22:34.765232+0000","last_fullsized":"2024-09-12T07:22:34.765232+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:58.776762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"166'781","reported_seq":3325,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764213+0000","last_change":"2024-09-12T07:22:21.150766+0000","last_active":"2024-09-12T07:22:34.764213+0000","last_peered":"2024-09-12T07:22:34.764213+0000","last_clean":"2024-09-12T07:22:34.764213+0000","last_became_active":"2024-09-12T07:22:21.147758+0000","last_became_peered":"2024-09-12T07:22:21.147758+0000","last_unstale":"2024-09-12T07:22:34.764213+0000","last_undegraded":"2024-09-12T07:22:34.764213+0000","last_fullsized":"2024-09-12T07:22:34.764213+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:15.197467+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.033858052999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5450,"num_read_kb":11647,"num_write":1052,"num_write_kb":13386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":9483314,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":299,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249699+0000","last_change":"2024-09-12T07:22:09.183569+0000","last_active":"2024-09-12T07:22:34.249699+0000","last_peered":"2024-09-12T07:22:34.249699+0000","last_clean":"2024-09-12T07:22:34.249699+0000","last_became_active":"2024-09-12T07:22:09.182547+0000","last_became_peered":"2024-09-12T07:22:09.182547+0000","last_unstale":"2024-09-12T07:22:34.249699+0000","last_undegraded":"2024-09-12T07:22:34.249699+0000","last_fullsized":"2024-09-12T07:22:34.249699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:52.520157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"162'406","reported_seq":10545,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264069+0000","last_change":"2024-09-12T07:22:06.195337+0000","last_active":"2024-09-12T07:22:34.264069+0000","last_peered":"2024-09-12T07:22:34.264069+0000","last_clean":"2024-09-12T07:22:34.264069+0000","last_became_active":"2024-09-12T07:22:01.146421+0000","last_became_peered":"2024-09-12T07:22:01.146421+0000","last_unstale":"2024-09-12T07:22:34.264069+0000","last_undegraded":"2024-09-12T07:22:34.264069+0000","last_fullsized":"2024-09-12T07:22:34.264069+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:38.812988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094442799999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11174,"num_read_kb":59795,"num_write":376,"num_write_kb":27716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":22543830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":360,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764760+0000","last_change":"2024-09-12T07:22:18.171696+0000","last_active":"2024-09-12T07:22:34.764760+0000","last_peered":"2024-09-12T07:22:34.764760+0000","last_clean":"2024-09-12T07:22:34.764760+0000","last_became_active":"2024-09-12T07:22:18.171266+0000","last_became_peered":"2024-09-12T07:22:18.171266+0000","last_unstale":"2024-09-12T07:22:34.764760+0000","last_undegraded":"2024-09-12T07:22:34.764760+0000","last_fullsized":"2024-09-12T07:22:34.764760+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:26.361507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"162'1062","reported_seq":6945,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707508+0000","last_change":"2024-09-12T07:22:22.152024+0000","last_active":"2024-09-12T07:22:34.707508+0000","last_peered":"2024-09-12T07:22:34.707508+0000","last_clean":"2024-09-12T07:22:34.707508+0000","last_became_active":"2024-09-12T07:22:22.150316+0000","last_became_peered":"2024-09-12T07:22:22.150316+0000","last_unstale":"2024-09-12T07:22:34.707508+0000","last_undegraded":"2024-09-12T07:22:34.707508+0000","last_fullsized":"2024-09-12T07:22:34.707508+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:57:13.384646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021189680000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8933,"num_read_kb":59814,"num_write":1570,"num_write_kb":27462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":14379761,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":45,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764860+0000","last_change":"2024-09-12T07:22:19.178038+0000","last_active":"2024-09-12T07:22:34.764860+0000","last_peered":"2024-09-12T07:22:34.764860+0000","last_clean":"2024-09-12T07:22:34.764860+0000","last_became_active":"2024-09-12T07:22:19.170715+0000","last_became_peered":"2024-09-12T07:22:19.170715+0000","last_unstale":"2024-09-12T07:22:34.764860+0000","last_undegraded":"2024-09-12T07:22:34.764860+0000","last_fullsized":"2024-09-12T07:22:34.764860+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:28.982893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":439,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764900+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:22:34.764900+0000","last_peered":"2024-09-12T07:22:34.764900+0000","last_clean":"2024-09-12T07:22:34.764900+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:22:34.764900+0000","last_undegraded":"2024-09-12T07:22:34.764900+0000","last_fullsized":"2024-09-12T07:22:34.764900+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"162'962","reported_seq":3107,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707372+0000","last_change":"2024-09-12T07:22:21.151517+0000","last_active":"2024-09-12T07:22:34.707372+0000","last_peered":"2024-09-12T07:22:34.707372+0000","last_clean":"2024-09-12T07:22:34.707372+0000","last_became_active":"2024-09-12T07:22:21.150083+0000","last_became_peered":"2024-09-12T07:22:21.150083+0000","last_unstale":"2024-09-12T07:22:34.707372+0000","last_undegraded":"2024-09-12T07:22:34.707372+0000","last_fullsized":"2024-09-12T07:22:34.707372+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":962,"log_dups_size":0,"ondisk_log_size":962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:38.471826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011038020000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5044,"num_read_kb":23221,"num_write":1448,"num_write_kb":24489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":21462395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"163'549","reported_seq":5426,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249824+0000","last_change":"2024-09-12T07:22:27.215048+0000","last_active":"2024-09-12T07:22:34.249824+0000","last_peered":"2024-09-12T07:22:34.249824+0000","last_clean":"2024-09-12T07:22:34.249824+0000","last_became_active":"2024-09-12T07:22:27.212942+0000","last_became_peered":"2024-09-12T07:22:27.212942+0000","last_unstale":"2024-09-12T07:22:34.249824+0000","last_undegraded":"2024-09-12T07:22:34.249824+0000","last_fullsized":"2024-09-12T07:22:34.249824+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:59.386875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021244950000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7563,"num_read_kb":32357,"num_write":525,"num_write_kb":20455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1710555,"num_keys_recovered":41,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"162'540","reported_seq":6677,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249766+0000","last_change":"2024-09-12T07:22:27.215838+0000","last_active":"2024-09-12T07:22:34.249766+0000","last_peered":"2024-09-12T07:22:34.249766+0000","last_clean":"2024-09-12T07:22:34.249766+0000","last_became_active":"2024-09-12T07:22:27.213533+0000","last_became_peered":"2024-09-12T07:22:27.213533+0000","last_unstale":"2024-09-12T07:22:34.249766+0000","last_undegraded":"2024-09-12T07:22:34.249766+0000","last_fullsized":"2024-09-12T07:22:34.249766+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:38.800561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069883599999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":6247,"num_read_kb":49000,"num_write":522,"num_write_kb":25535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":42565291,"num_keys_recovered":104,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"162'692","reported_seq":3567,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707411+0000","last_change":"2024-09-12T07:22:06.173724+0000","last_active":"2024-09-12T07:22:34.707411+0000","last_peered":"2024-09-12T07:22:34.707411+0000","last_clean":"2024-09-12T07:22:34.707411+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:22:34.707411+0000","last_undegraded":"2024-09-12T07:22:34.707411+0000","last_fullsized":"2024-09-12T07:22:34.707411+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:30.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015261610000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6009,"num_read_kb":21069,"num_write":727,"num_write_kb":17071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"162'445","reported_seq":9922,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764933+0000","last_change":"2024-09-12T07:22:22.151400+0000","last_active":"2024-09-12T07:22:34.764933+0000","last_peered":"2024-09-12T07:22:34.764933+0000","last_clean":"2024-09-12T07:22:34.764933+0000","last_became_active":"2024-09-12T07:22:22.149715+0000","last_became_peered":"2024-09-12T07:22:22.149715+0000","last_unstale":"2024-09-12T07:22:34.764933+0000","last_undegraded":"2024-09-12T07:22:34.764933+0000","last_fullsized":"2024-09-12T07:22:34.764933+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:51.662782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016506240000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11019,"num_read_kb":49685,"num_write":382,"num_write_kb":15924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":18973449,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"161'769","reported_seq":3640,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765503+0000","last_change":"2024-09-12T07:22:22.153946+0000","last_active":"2024-09-12T07:22:34.765503+0000","last_peered":"2024-09-12T07:22:34.765503+0000","last_clean":"2024-09-12T07:22:34.765503+0000","last_became_active":"2024-09-12T07:22:22.152455+0000","last_became_peered":"2024-09-12T07:22:22.152455+0000","last_unstale":"2024-09-12T07:22:34.765503+0000","last_undegraded":"2024-09-12T07:22:34.765503+0000","last_fullsized":"2024-09-12T07:22:34.765503+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:17.209337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014515540000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5118,"num_read_kb":30709,"num_write":1176,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":7370075,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"162'531","reported_seq":5409,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765193+0000","last_change":"2024-09-12T07:22:06.166758+0000","last_active":"2024-09-12T07:22:34.765193+0000","last_peered":"2024-09-12T07:22:34.765193+0000","last_clean":"2024-09-12T07:22:34.765193+0000","last_became_active":"2024-09-12T07:22:00.145108+0000","last_became_peered":"2024-09-12T07:22:00.145108+0000","last_unstale":"2024-09-12T07:22:34.765193+0000","last_undegraded":"2024-09-12T07:22:34.765193+0000","last_fullsized":"2024-09-12T07:22:34.765193+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:31.569737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801299999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6080,"num_read_kb":38867,"num_write":581,"num_write_kb":16891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":10508628,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"162'596","reported_seq":3023,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765441+0000","last_change":"2024-09-12T07:22:06.166247+0000","last_active":"2024-09-12T07:22:34.765441+0000","last_peered":"2024-09-12T07:22:34.765441+0000","last_clean":"2024-09-12T07:22:34.765441+0000","last_became_active":"2024-09-12T07:22:01.143359+0000","last_became_peered":"2024-09-12T07:22:01.143359+0000","last_unstale":"2024-09-12T07:22:34.765441+0000","last_undegraded":"2024-09-12T07:22:34.765441+0000","last_fullsized":"2024-09-12T07:22:34.765441+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:49.338904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001515553,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3463,"num_read_kb":13505,"num_write":637,"num_write_kb":16522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":8725508,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"152'338","reported_seq":1797,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765144+0000","last_change":"2024-09-12T07:22:31.247099+0000","last_active":"2024-09-12T07:22:34.765144+0000","last_peered":"2024-09-12T07:22:34.765144+0000","last_clean":"2024-09-12T07:22:34.765144+0000","last_became_active":"2024-09-12T07:22:31.245905+0000","last_became_peered":"2024-09-12T07:22:31.245905+0000","last_unstale":"2024-09-12T07:22:34.765144+0000","last_undegraded":"2024-09-12T07:22:34.765144+0000","last_fullsized":"2024-09-12T07:22:34.765144+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:10.350713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816669999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1472,"num_read_kb":27101,"num_write":327,"num_write_kb":18963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"163'454","reported_seq":7380,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764987+0000","last_change":"2024-09-12T07:22:06.164601+0000","last_active":"2024-09-12T07:22:34.764987+0000","last_peered":"2024-09-12T07:22:34.764987+0000","last_clean":"2024-09-12T07:22:34.764987+0000","last_became_active":"2024-09-12T07:22:01.143631+0000","last_became_peered":"2024-09-12T07:22:01.143631+0000","last_unstale":"2024-09-12T07:22:34.764987+0000","last_undegraded":"2024-09-12T07:22:34.764987+0000","last_fullsized":"2024-09-12T07:22:34.764987+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:34.629049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065451099999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7090,"num_read_kb":47369,"num_write":461,"num_write_kb":21538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":17766842,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"159'408","reported_seq":11210,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264074+0000","last_change":"2024-09-12T07:22:06.196502+0000","last_active":"2024-09-12T07:22:34.264074+0000","last_peered":"2024-09-12T07:22:34.264074+0000","last_clean":"2024-09-12T07:22:34.264074+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:22:34.264074+0000","last_undegraded":"2024-09-12T07:22:34.264074+0000","last_fullsized":"2024-09-12T07:22:34.264074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:48.940535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019415999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11863,"num_read_kb":47954,"num_write":410,"num_write_kb":15586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'476","reported_seq":19728,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765222+0000","last_change":"2024-09-12T07:22:06.165173+0000","last_active":"2024-09-12T07:22:34.765222+0000","last_peered":"2024-09-12T07:22:34.765222+0000","last_clean":"2024-09-12T07:22:34.765222+0000","last_became_active":"2024-09-12T07:22:01.143577+0000","last_became_peered":"2024-09-12T07:22:01.143577+0000","last_unstale":"2024-09-12T07:22:34.765222+0000","last_undegraded":"2024-09-12T07:22:34.765222+0000","last_fullsized":"2024-09-12T07:22:34.765222+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:12:53.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010515870000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":18947,"num_read_kb":82796,"num_write":553,"num_write_kb":44274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":53090062,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":323,"num_object_clones":0,"num_object_copies":646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":319,"num_read":260585,"num_read_kb":1190964,"num_write":20483,"num_write_kb":597347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5845,"num_bytes_recovered":432794818,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},"log_size":17959,"ondisk_log_size":17959,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56164,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315676,"statfs":{"total":96636764160,"available":96579252224,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33376,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338464,"statfs":{"total":96636764160,"available":96602587136,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55576,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316264,"statfs":{"total":96636764160,"available":96579854336,"internally_reserved":0,"allocated":2232320,"data_stored":1075567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51432,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320408,"statfs":{"total":96636764160,"available":96584097792,"internally_reserved":0,"allocated":2244608,"data_stored":1081409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69480,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302360,"statfs":{"total":96636764160,"available":96565616640,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":165,"seq":708669603848,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31008,"kb_used_data":2188,"kb_used_omap":23,"kb_used_meta":28776,"kb_avail":94340832,"statfs":{"total":96636764160,"available":96605011968,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24213,"internal_metadata":29466987},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55592,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316248,"statfs":{"total":96636764160,"available":96579837952,"internally_reserved":0,"allocated":2248704,"data_stored":1081428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64488,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307352,"statfs":{"total":96636764160,"available":96570728448,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:22:38.159 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-12T07:22:38.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-12T07:22:38.479 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-12T07:22:38.493 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T07:22:38.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:22:38.795 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:22:38.796 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:22:38.807 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-12T07:22:37.525611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":327,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":319,"num_read":260651,"num_read_kb":1191019,"num_write":20586,"num_write_kb":598977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5849,"num_bytes_recovered":432794875,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18034,"ondisk_log_size":18034,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417116,"kb_used_data":18668,"kb_used_omap":34,"kb_used_meta":398237,"kb_avail":754557604,"statfs":{"total":773094113280,"available":772666986496,"internally_reserved":0,"allocated":19116032,"data_stored":9826090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35370,"internal_metadata":407795158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001526"},"pg_stats":[{"pgid":"3.1f","version":"166'638","reported_seq":6785,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264251+0000","last_change":"2024-09-12T07:22:22.149919+0000","last_active":"2024-09-12T07:22:34.264251+0000","last_peered":"2024-09-12T07:22:34.264251+0000","last_clean":"2024-09-12T07:22:34.264251+0000","last_became_active":"2024-09-12T07:22:22.148401+0000","last_became_peered":"2024-09-12T07:22:22.148401+0000","last_unstale":"2024-09-12T07:22:34.264251+0000","last_undegraded":"2024-09-12T07:22:34.264251+0000","last_fullsized":"2024-09-12T07:22:34.264251+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:27:39.529993+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.029437784000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7771,"num_read_kb":29649,"num_write":783,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":2971697,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"161'489","reported_seq":2632,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707535+0000","last_change":"2024-09-12T07:22:06.175231+0000","last_active":"2024-09-12T07:22:34.707535+0000","last_peered":"2024-09-12T07:22:34.707535+0000","last_clean":"2024-09-12T07:22:34.707535+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:22:34.707535+0000","last_undegraded":"2024-09-12T07:22:34.707535+0000","last_fullsized":"2024-09-12T07:22:34.707535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:33:19.436850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026801759999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20338,"num_read_kb":38377,"num_write":424,"num_write_kb":22354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"163'468","reported_seq":21373,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765348+0000","last_change":"2024-09-12T07:22:22.151697+0000","last_active":"2024-09-12T07:22:34.765348+0000","last_peered":"2024-09-12T07:22:34.765348+0000","last_clean":"2024-09-12T07:22:34.765348+0000","last_became_active":"2024-09-12T07:22:22.149491+0000","last_became_peered":"2024-09-12T07:22:22.149491+0000","last_unstale":"2024-09-12T07:22:34.765348+0000","last_undegraded":"2024-09-12T07:22:34.765348+0000","last_fullsized":"2024-09-12T07:22:34.765348+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:51.871543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010058409999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":20924,"num_read_kb":39045,"num_write":507,"num_write_kb":15255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":23048347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"163'607","reported_seq":3893,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764874+0000","last_change":"2024-09-12T07:22:21.150401+0000","last_active":"2024-09-12T07:22:34.764874+0000","last_peered":"2024-09-12T07:22:34.764874+0000","last_clean":"2024-09-12T07:22:34.764874+0000","last_became_active":"2024-09-12T07:22:21.148942+0000","last_became_peered":"2024-09-12T07:22:21.148942+0000","last_unstale":"2024-09-12T07:22:34.764874+0000","last_undegraded":"2024-09-12T07:22:34.764874+0000","last_fullsized":"2024-09-12T07:22:34.764874+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:45.508340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014959909999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":3878,"num_read_kb":17362,"num_write":791,"num_write_kb":11625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":8199265,"num_keys_recovered":174,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"162'447","reported_seq":22008,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264233+0000","last_change":"2024-09-12T07:22:06.195928+0000","last_active":"2024-09-12T07:22:34.264233+0000","last_peered":"2024-09-12T07:22:34.264233+0000","last_clean":"2024-09-12T07:22:34.264233+0000","last_became_active":"2024-09-12T07:22:01.145404+0000","last_became_peered":"2024-09-12T07:22:01.145404+0000","last_unstale":"2024-09-12T07:22:34.264233+0000","last_undegraded":"2024-09-12T07:22:34.264233+0000","last_fullsized":"2024-09-12T07:22:34.264233+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:03.385436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014847230000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":21866,"num_read_kb":61051,"num_write":426,"num_write_kb":15720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":12377120,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'388","reported_seq":4868,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249862+0000","last_change":"2024-09-12T07:22:27.215825+0000","last_active":"2024-09-12T07:22:34.249862+0000","last_peered":"2024-09-12T07:22:34.249862+0000","last_clean":"2024-09-12T07:22:34.249862+0000","last_became_active":"2024-09-12T07:22:27.213519+0000","last_became_peered":"2024-09-12T07:22:27.213519+0000","last_unstale":"2024-09-12T07:22:34.249862+0000","last_undegraded":"2024-09-12T07:22:34.249862+0000","last_fullsized":"2024-09-12T07:22:34.249862+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.924110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022138280000000001,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4657,"num_read_kb":13211,"num_write":331,"num_write_kb":17325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":27687211,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"163'649","reported_seq":7971,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764092+0000","last_change":"2024-09-12T07:22:21.149665+0000","last_active":"2024-09-12T07:22:34.764092+0000","last_peered":"2024-09-12T07:22:34.764092+0000","last_clean":"2024-09-12T07:22:34.764092+0000","last_became_active":"2024-09-12T07:22:21.147583+0000","last_became_peered":"2024-09-12T07:22:21.147583+0000","last_unstale":"2024-09-12T07:22:34.764092+0000","last_undegraded":"2024-09-12T07:22:34.764092+0000","last_fullsized":"2024-09-12T07:22:34.764092+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:22.335622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023936500000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8436,"num_read_kb":50796,"num_write":779,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":7815100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"163'609","reported_seq":3818,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707368+0000","last_change":"2024-09-12T07:22:06.173059+0000","last_active":"2024-09-12T07:22:34.707368+0000","last_peered":"2024-09-12T07:22:34.707368+0000","last_clean":"2024-09-12T07:22:34.707368+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:22:34.707368+0000","last_undegraded":"2024-09-12T07:22:34.707368+0000","last_fullsized":"2024-09-12T07:22:34.707368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:17.727732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099897500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6008,"num_read_kb":52191,"num_write":744,"num_write_kb":22227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"162'653","reported_seq":7128,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264260+0000","last_change":"2024-09-12T07:22:22.149915+0000","last_active":"2024-09-12T07:22:34.264260+0000","last_peered":"2024-09-12T07:22:34.264260+0000","last_clean":"2024-09-12T07:22:34.264260+0000","last_became_active":"2024-09-12T07:22:22.148392+0000","last_became_peered":"2024-09-12T07:22:22.148392+0000","last_unstale":"2024-09-12T07:22:34.264260+0000","last_undegraded":"2024-09-12T07:22:34.264260+0000","last_fullsized":"2024-09-12T07:22:34.264260+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:29.255739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002496505,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8906,"num_read_kb":19832,"num_write":605,"num_write_kb":22990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":20211745,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"163'375","reported_seq":1927,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765128+0000","last_change":"2024-09-12T07:22:21.149525+0000","last_active":"2024-09-12T07:22:34.765128+0000","last_peered":"2024-09-12T07:22:34.765128+0000","last_clean":"2024-09-12T07:22:34.765128+0000","last_became_active":"2024-09-12T07:22:21.148026+0000","last_became_peered":"2024-09-12T07:22:21.148026+0000","last_unstale":"2024-09-12T07:22:34.765128+0000","last_undegraded":"2024-09-12T07:22:34.765128+0000","last_fullsized":"2024-09-12T07:22:34.765128+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:15.887884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017138209999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1348,"num_read_kb":18539,"num_write":421,"num_write_kb":14413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":10121363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"152'344","reported_seq":7171,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765014+0000","last_change":"2024-09-12T07:22:06.164819+0000","last_active":"2024-09-12T07:22:34.765014+0000","last_peered":"2024-09-12T07:22:34.765014+0000","last_clean":"2024-09-12T07:22:34.765014+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:22:34.765014+0000","last_undegraded":"2024-09-12T07:22:34.765014+0000","last_fullsized":"2024-09-12T07:22:34.765014+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:45.370726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080767000000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7070,"num_read_kb":53310,"num_write":299,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"160'391","reported_seq":5850,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.250009+0000","last_change":"2024-09-12T07:22:27.214683+0000","last_active":"2024-09-12T07:22:34.250009+0000","last_peered":"2024-09-12T07:22:34.250009+0000","last_clean":"2024-09-12T07:22:34.250009+0000","last_became_active":"2024-09-12T07:22:27.212420+0000","last_became_peered":"2024-09-12T07:22:27.212420+0000","last_unstale":"2024-09-12T07:22:34.250009+0000","last_undegraded":"2024-09-12T07:22:34.250009+0000","last_fullsized":"2024-09-12T07:22:34.250009+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:16.234305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097451600000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5380,"num_read_kb":60804,"num_write":390,"num_write_kb":30252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":31149476,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764999+0000","last_change":"2024-09-12T07:22:31.308130+0000","last_active":"2024-09-12T07:22:34.764999+0000","last_peered":"2024-09-12T07:22:34.764999+0000","last_clean":"2024-09-12T07:22:34.764999+0000","last_became_active":"2024-09-12T07:22:31.307821+0000","last_became_peered":"2024-09-12T07:22:31.307821+0000","last_unstale":"2024-09-12T07:22:34.764999+0000","last_undegraded":"2024-09-12T07:22:34.764999+0000","last_fullsized":"2024-09-12T07:22:34.764999+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:45.276396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"163'639","reported_seq":6266,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264231+0000","last_change":"2024-09-12T07:22:21.153348+0000","last_active":"2024-09-12T07:22:34.264231+0000","last_peered":"2024-09-12T07:22:34.264231+0000","last_clean":"2024-09-12T07:22:34.264231+0000","last_became_active":"2024-09-12T07:22:21.152036+0000","last_became_peered":"2024-09-12T07:22:21.152036+0000","last_unstale":"2024-09-12T07:22:34.264231+0000","last_undegraded":"2024-09-12T07:22:34.264231+0000","last_fullsized":"2024-09-12T07:22:34.264231+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:16.901686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014037019999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6009,"num_read_kb":23012,"num_write":704,"num_write_kb":11415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":1480654,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764229+0000","last_change":"2024-09-12T07:22:18.173512+0000","last_active":"2024-09-12T07:22:34.764229+0000","last_peered":"2024-09-12T07:22:34.764229+0000","last_clean":"2024-09-12T07:22:34.764229+0000","last_became_active":"2024-09-12T07:22:18.173148+0000","last_became_peered":"2024-09-12T07:22:18.173148+0000","last_unstale":"2024-09-12T07:22:34.764229+0000","last_undegraded":"2024-09-12T07:22:34.764229+0000","last_fullsized":"2024-09-12T07:22:34.764229+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:07.135570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"162'679","reported_seq":2209,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707544+0000","last_change":"2024-09-12T07:22:06.174857+0000","last_active":"2024-09-12T07:22:34.707544+0000","last_peered":"2024-09-12T07:22:34.707544+0000","last_clean":"2024-09-12T07:22:34.707544+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:22:34.707544+0000","last_undegraded":"2024-09-12T07:22:34.707544+0000","last_fullsized":"2024-09-12T07:22:34.707544+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:16.561231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025712959999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3057,"num_read_kb":14289,"num_write":898,"num_write_kb":11329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":348,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764698+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:22:34.764698+0000","last_peered":"2024-09-12T07:22:34.764698+0000","last_clean":"2024-09-12T07:22:34.764698+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:22:34.764698+0000","last_undegraded":"2024-09-12T07:22:34.764698+0000","last_fullsized":"2024-09-12T07:22:34.764698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"166'506","reported_seq":3850,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764186+0000","last_change":"2024-09-12T07:22:21.149345+0000","last_active":"2024-09-12T07:22:34.764186+0000","last_peered":"2024-09-12T07:22:34.764186+0000","last_clean":"2024-09-12T07:22:34.764186+0000","last_became_active":"2024-09-12T07:22:21.147703+0000","last_became_peered":"2024-09-12T07:22:21.147703+0000","last_unstale":"2024-09-12T07:22:34.764186+0000","last_undegraded":"2024-09-12T07:22:34.764186+0000","last_fullsized":"2024-09-12T07:22:34.764186+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:05.437283+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.023667932999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4682,"num_read_kb":14844,"num_write":579,"num_write_kb":8813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6163290,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764154+0000","last_change":"2024-09-12T07:22:18.169631+0000","last_active":"2024-09-12T07:22:34.764154+0000","last_peered":"2024-09-12T07:22:34.764154+0000","last_clean":"2024-09-12T07:22:34.764154+0000","last_became_active":"2024-09-12T07:22:18.169285+0000","last_became_peered":"2024-09-12T07:22:18.169285+0000","last_unstale":"2024-09-12T07:22:34.764154+0000","last_undegraded":"2024-09-12T07:22:34.764154+0000","last_fullsized":"2024-09-12T07:22:34.764154+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:21.373190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"166'565","reported_seq":10840,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765459+0000","last_change":"2024-09-12T07:22:22.153784+0000","last_active":"2024-09-12T07:22:34.765459+0000","last_peered":"2024-09-12T07:22:34.765459+0000","last_clean":"2024-09-12T07:22:34.765459+0000","last_became_active":"2024-09-12T07:22:22.151818+0000","last_became_peered":"2024-09-12T07:22:22.151818+0000","last_unstale":"2024-09-12T07:22:34.765459+0000","last_undegraded":"2024-09-12T07:22:34.765459+0000","last_fullsized":"2024-09-12T07:22:34.765459+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:02.130727+0000","scrub_duration":0,"objects_trimmed":18,"snaptrim_duration":0.023276249999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11624,"num_read_kb":77562,"num_write":595,"num_write_kb":25411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":22089334,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"163'503","reported_seq":4495,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249801+0000","last_change":"2024-09-12T07:22:27.215301+0000","last_active":"2024-09-12T07:22:34.249801+0000","last_peered":"2024-09-12T07:22:34.249801+0000","last_clean":"2024-09-12T07:22:34.249801+0000","last_became_active":"2024-09-12T07:22:27.213073+0000","last_became_peered":"2024-09-12T07:22:27.213073+0000","last_unstale":"2024-09-12T07:22:34.249801+0000","last_undegraded":"2024-09-12T07:22:34.249801+0000","last_fullsized":"2024-09-12T07:22:34.249801+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:10.316262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018780750000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":3159,"num_read_kb":12201,"num_write":460,"num_write_kb":10579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":2603172,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":48,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765232+0000","last_change":"2024-09-12T07:22:19.200232+0000","last_active":"2024-09-12T07:22:34.765232+0000","last_peered":"2024-09-12T07:22:34.765232+0000","last_clean":"2024-09-12T07:22:34.765232+0000","last_became_active":"2024-09-12T07:22:19.178595+0000","last_became_peered":"2024-09-12T07:22:19.178595+0000","last_unstale":"2024-09-12T07:22:34.765232+0000","last_undegraded":"2024-09-12T07:22:34.765232+0000","last_fullsized":"2024-09-12T07:22:34.765232+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:58.776762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"166'781","reported_seq":3325,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764213+0000","last_change":"2024-09-12T07:22:21.150766+0000","last_active":"2024-09-12T07:22:34.764213+0000","last_peered":"2024-09-12T07:22:34.764213+0000","last_clean":"2024-09-12T07:22:34.764213+0000","last_became_active":"2024-09-12T07:22:21.147758+0000","last_became_peered":"2024-09-12T07:22:21.147758+0000","last_unstale":"2024-09-12T07:22:34.764213+0000","last_undegraded":"2024-09-12T07:22:34.764213+0000","last_fullsized":"2024-09-12T07:22:34.764213+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:15.197467+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.033858052999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5450,"num_read_kb":11647,"num_write":1052,"num_write_kb":13386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":9483314,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":299,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249699+0000","last_change":"2024-09-12T07:22:09.183569+0000","last_active":"2024-09-12T07:22:34.249699+0000","last_peered":"2024-09-12T07:22:34.249699+0000","last_clean":"2024-09-12T07:22:34.249699+0000","last_became_active":"2024-09-12T07:22:09.182547+0000","last_became_peered":"2024-09-12T07:22:09.182547+0000","last_unstale":"2024-09-12T07:22:34.249699+0000","last_undegraded":"2024-09-12T07:22:34.249699+0000","last_fullsized":"2024-09-12T07:22:34.249699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:52.520157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"162'406","reported_seq":10545,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264069+0000","last_change":"2024-09-12T07:22:06.195337+0000","last_active":"2024-09-12T07:22:34.264069+0000","last_peered":"2024-09-12T07:22:34.264069+0000","last_clean":"2024-09-12T07:22:34.264069+0000","last_became_active":"2024-09-12T07:22:01.146421+0000","last_became_peered":"2024-09-12T07:22:01.146421+0000","last_unstale":"2024-09-12T07:22:34.264069+0000","last_undegraded":"2024-09-12T07:22:34.264069+0000","last_fullsized":"2024-09-12T07:22:34.264069+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:38.812988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094442799999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11174,"num_read_kb":59795,"num_write":376,"num_write_kb":27716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":22543830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":360,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764760+0000","last_change":"2024-09-12T07:22:18.171696+0000","last_active":"2024-09-12T07:22:34.764760+0000","last_peered":"2024-09-12T07:22:34.764760+0000","last_clean":"2024-09-12T07:22:34.764760+0000","last_became_active":"2024-09-12T07:22:18.171266+0000","last_became_peered":"2024-09-12T07:22:18.171266+0000","last_unstale":"2024-09-12T07:22:34.764760+0000","last_undegraded":"2024-09-12T07:22:34.764760+0000","last_fullsized":"2024-09-12T07:22:34.764760+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:26.361507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"162'1062","reported_seq":6945,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707508+0000","last_change":"2024-09-12T07:22:22.152024+0000","last_active":"2024-09-12T07:22:34.707508+0000","last_peered":"2024-09-12T07:22:34.707508+0000","last_clean":"2024-09-12T07:22:34.707508+0000","last_became_active":"2024-09-12T07:22:22.150316+0000","last_became_peered":"2024-09-12T07:22:22.150316+0000","last_unstale":"2024-09-12T07:22:34.707508+0000","last_undegraded":"2024-09-12T07:22:34.707508+0000","last_fullsized":"2024-09-12T07:22:34.707508+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:57:13.384646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021189680000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8933,"num_read_kb":59814,"num_write":1570,"num_write_kb":27462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":14379761,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":45,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764860+0000","last_change":"2024-09-12T07:22:19.178038+0000","last_active":"2024-09-12T07:22:34.764860+0000","last_peered":"2024-09-12T07:22:34.764860+0000","last_clean":"2024-09-12T07:22:34.764860+0000","last_became_active":"2024-09-12T07:22:19.170715+0000","last_became_peered":"2024-09-12T07:22:19.170715+0000","last_unstale":"2024-09-12T07:22:34.764860+0000","last_undegraded":"2024-09-12T07:22:34.764860+0000","last_fullsized":"2024-09-12T07:22:34.764860+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:28.982893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":439,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764900+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:22:34.764900+0000","last_peered":"2024-09-12T07:22:34.764900+0000","last_clean":"2024-09-12T07:22:34.764900+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:22:34.764900+0000","last_undegraded":"2024-09-12T07:22:34.764900+0000","last_fullsized":"2024-09-12T07:22:34.764900+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"162'962","reported_seq":3107,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707372+0000","last_change":"2024-09-12T07:22:21.151517+0000","last_active":"2024-09-12T07:22:34.707372+0000","last_peered":"2024-09-12T07:22:34.707372+0000","last_clean":"2024-09-12T07:22:34.707372+0000","last_became_active":"2024-09-12T07:22:21.150083+0000","last_became_peered":"2024-09-12T07:22:21.150083+0000","last_unstale":"2024-09-12T07:22:34.707372+0000","last_undegraded":"2024-09-12T07:22:34.707372+0000","last_fullsized":"2024-09-12T07:22:34.707372+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":962,"log_dups_size":0,"ondisk_log_size":962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:38.471826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011038020000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5044,"num_read_kb":23221,"num_write":1448,"num_write_kb":24489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":21462395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"163'549","reported_seq":5426,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249824+0000","last_change":"2024-09-12T07:22:27.215048+0000","last_active":"2024-09-12T07:22:34.249824+0000","last_peered":"2024-09-12T07:22:34.249824+0000","last_clean":"2024-09-12T07:22:34.249824+0000","last_became_active":"2024-09-12T07:22:27.212942+0000","last_became_peered":"2024-09-12T07:22:27.212942+0000","last_unstale":"2024-09-12T07:22:34.249824+0000","last_undegraded":"2024-09-12T07:22:34.249824+0000","last_fullsized":"2024-09-12T07:22:34.249824+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:59.386875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021244950000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7563,"num_read_kb":32357,"num_write":525,"num_write_kb":20455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1710555,"num_keys_recovered":41,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"162'540","reported_seq":6677,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249766+0000","last_change":"2024-09-12T07:22:27.215838+0000","last_active":"2024-09-12T07:22:34.249766+0000","last_peered":"2024-09-12T07:22:34.249766+0000","last_clean":"2024-09-12T07:22:34.249766+0000","last_became_active":"2024-09-12T07:22:27.213533+0000","last_became_peered":"2024-09-12T07:22:27.213533+0000","last_unstale":"2024-09-12T07:22:34.249766+0000","last_undegraded":"2024-09-12T07:22:34.249766+0000","last_fullsized":"2024-09-12T07:22:34.249766+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:38.800561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069883599999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":6247,"num_read_kb":49000,"num_write":522,"num_write_kb":25535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":42565291,"num_keys_recovered":104,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"162'692","reported_seq":3567,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707411+0000","last_change":"2024-09-12T07:22:06.173724+0000","last_active":"2024-09-12T07:22:34.707411+0000","last_peered":"2024-09-12T07:22:34.707411+0000","last_clean":"2024-09-12T07:22:34.707411+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:22:34.707411+0000","last_undegraded":"2024-09-12T07:22:34.707411+0000","last_fullsized":"2024-09-12T07:22:34.707411+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:30.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015261610000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6009,"num_read_kb":21069,"num_write":727,"num_write_kb":17071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"162'445","reported_seq":9922,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764933+0000","last_change":"2024-09-12T07:22:22.151400+0000","last_active":"2024-09-12T07:22:34.764933+0000","last_peered":"2024-09-12T07:22:34.764933+0000","last_clean":"2024-09-12T07:22:34.764933+0000","last_became_active":"2024-09-12T07:22:22.149715+0000","last_became_peered":"2024-09-12T07:22:22.149715+0000","last_unstale":"2024-09-12T07:22:34.764933+0000","last_undegraded":"2024-09-12T07:22:34.764933+0000","last_fullsized":"2024-09-12T07:22:34.764933+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:51.662782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016506240000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11019,"num_read_kb":49685,"num_write":382,"num_write_kb":15924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":18973449,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"161'769","reported_seq":3640,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765503+0000","last_change":"2024-09-12T07:22:22.153946+0000","last_active":"2024-09-12T07:22:34.765503+0000","last_peered":"2024-09-12T07:22:34.765503+0000","last_clean":"2024-09-12T07:22:34.765503+0000","last_became_active":"2024-09-12T07:22:22.152455+0000","last_became_peered":"2024-09-12T07:22:22.152455+0000","last_unstale":"2024-09-12T07:22:34.765503+0000","last_undegraded":"2024-09-12T07:22:34.765503+0000","last_fullsized":"2024-09-12T07:22:34.765503+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:17.209337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014515540000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5118,"num_read_kb":30709,"num_write":1176,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":7370075,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"162'531","reported_seq":5409,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765193+0000","last_change":"2024-09-12T07:22:06.166758+0000","last_active":"2024-09-12T07:22:34.765193+0000","last_peered":"2024-09-12T07:22:34.765193+0000","last_clean":"2024-09-12T07:22:34.765193+0000","last_became_active":"2024-09-12T07:22:00.145108+0000","last_became_peered":"2024-09-12T07:22:00.145108+0000","last_unstale":"2024-09-12T07:22:34.765193+0000","last_undegraded":"2024-09-12T07:22:34.765193+0000","last_fullsized":"2024-09-12T07:22:34.765193+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:31.569737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801299999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6080,"num_read_kb":38867,"num_write":581,"num_write_kb":16891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":10508628,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"162'596","reported_seq":3023,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765441+0000","last_change":"2024-09-12T07:22:06.166247+0000","last_active":"2024-09-12T07:22:34.765441+0000","last_peered":"2024-09-12T07:22:34.765441+0000","last_clean":"2024-09-12T07:22:34.765441+0000","last_became_active":"2024-09-12T07:22:01.143359+0000","last_became_peered":"2024-09-12T07:22:01.143359+0000","last_unstale":"2024-09-12T07:22:34.765441+0000","last_undegraded":"2024-09-12T07:22:34.765441+0000","last_fullsized":"2024-09-12T07:22:34.765441+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:49.338904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001515553,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3463,"num_read_kb":13505,"num_write":637,"num_write_kb":16522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":8725508,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"152'338","reported_seq":1797,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765144+0000","last_change":"2024-09-12T07:22:31.247099+0000","last_active":"2024-09-12T07:22:34.765144+0000","last_peered":"2024-09-12T07:22:34.765144+0000","last_clean":"2024-09-12T07:22:34.765144+0000","last_became_active":"2024-09-12T07:22:31.245905+0000","last_became_peered":"2024-09-12T07:22:31.245905+0000","last_unstale":"2024-09-12T07:22:34.765144+0000","last_undegraded":"2024-09-12T07:22:34.765144+0000","last_fullsized":"2024-09-12T07:22:34.765144+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:10.350713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816669999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1472,"num_read_kb":27101,"num_write":327,"num_write_kb":18963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"163'454","reported_seq":7380,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764987+0000","last_change":"2024-09-12T07:22:06.164601+0000","last_active":"2024-09-12T07:22:34.764987+0000","last_peered":"2024-09-12T07:22:34.764987+0000","last_clean":"2024-09-12T07:22:34.764987+0000","last_became_active":"2024-09-12T07:22:01.143631+0000","last_became_peered":"2024-09-12T07:22:01.143631+0000","last_unstale":"2024-09-12T07:22:34.764987+0000","last_undegraded":"2024-09-12T07:22:34.764987+0000","last_fullsized":"2024-09-12T07:22:34.764987+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:34.629049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065451099999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7090,"num_read_kb":47369,"num_write":461,"num_write_kb":21538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":17766842,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"159'408","reported_seq":11210,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264074+0000","last_change":"2024-09-12T07:22:06.196502+0000","last_active":"2024-09-12T07:22:34.264074+0000","last_peered":"2024-09-12T07:22:34.264074+0000","last_clean":"2024-09-12T07:22:34.264074+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:22:34.264074+0000","last_undegraded":"2024-09-12T07:22:34.264074+0000","last_fullsized":"2024-09-12T07:22:34.264074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:48.940535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019415999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11863,"num_read_kb":47954,"num_write":410,"num_write_kb":15586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'476","reported_seq":19728,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765222+0000","last_change":"2024-09-12T07:22:06.165173+0000","last_active":"2024-09-12T07:22:34.765222+0000","last_peered":"2024-09-12T07:22:34.765222+0000","last_clean":"2024-09-12T07:22:34.765222+0000","last_became_active":"2024-09-12T07:22:01.143577+0000","last_became_peered":"2024-09-12T07:22:01.143577+0000","last_unstale":"2024-09-12T07:22:34.765222+0000","last_undegraded":"2024-09-12T07:22:34.765222+0000","last_fullsized":"2024-09-12T07:22:34.765222+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:12:53.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010515870000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":18947,"num_read_kb":82796,"num_write":553,"num_write_kb":44274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":53090062,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":323,"num_object_clones":0,"num_object_copies":646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":319,"num_read":260585,"num_read_kb":1190964,"num_write":20483,"num_write_kb":597347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5845,"num_bytes_recovered":432794818,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},"log_size":17959,"ondisk_log_size":17959,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56164,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315676,"statfs":{"total":96636764160,"available":96579252224,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33376,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338464,"statfs":{"total":96636764160,"available":96602587136,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55576,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316264,"statfs":{"total":96636764160,"available":96579854336,"internally_reserved":0,"allocated":2232320,"data_stored":1075567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51432,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320408,"statfs":{"total":96636764160,"available":96584097792,"internally_reserved":0,"allocated":2244608,"data_stored":1081409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69480,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302360,"statfs":{"total":96636764160,"available":96565616640,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":165,"seq":708669603848,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31008,"kb_used_data":2188,"kb_used_omap":23,"kb_used_meta":28776,"kb_avail":94340832,"statfs":{"total":96636764160,"available":96605011968,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24213,"internal_metadata":29466987},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55592,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316248,"statfs":{"total":96636764160,"available":96579837952,"internally_reserved":0,"allocated":2248704,"data_stored":1081428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64488,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307352,"statfs":{"total":96636764160,"available":96570728448,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:22:38.810 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:22:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:22:39.110 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:22:39.120 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-12T07:22:37.525611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":327,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":319,"num_read":260651,"num_read_kb":1191019,"num_write":20586,"num_write_kb":598977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5849,"num_bytes_recovered":432794875,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18034,"ondisk_log_size":18034,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417116,"kb_used_data":18668,"kb_used_omap":34,"kb_used_meta":398237,"kb_avail":754557604,"statfs":{"total":773094113280,"available":772666986496,"internally_reserved":0,"allocated":19116032,"data_stored":9826090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35370,"internal_metadata":407795158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001526"},"pg_stats":[{"pgid":"3.1f","version":"166'638","reported_seq":6785,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264251+0000","last_change":"2024-09-12T07:22:22.149919+0000","last_active":"2024-09-12T07:22:34.264251+0000","last_peered":"2024-09-12T07:22:34.264251+0000","last_clean":"2024-09-12T07:22:34.264251+0000","last_became_active":"2024-09-12T07:22:22.148401+0000","last_became_peered":"2024-09-12T07:22:22.148401+0000","last_unstale":"2024-09-12T07:22:34.264251+0000","last_undegraded":"2024-09-12T07:22:34.264251+0000","last_fullsized":"2024-09-12T07:22:34.264251+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:27:39.529993+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.029437784000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7771,"num_read_kb":29649,"num_write":783,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":2971697,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"161'489","reported_seq":2632,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707535+0000","last_change":"2024-09-12T07:22:06.175231+0000","last_active":"2024-09-12T07:22:34.707535+0000","last_peered":"2024-09-12T07:22:34.707535+0000","last_clean":"2024-09-12T07:22:34.707535+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:22:34.707535+0000","last_undegraded":"2024-09-12T07:22:34.707535+0000","last_fullsized":"2024-09-12T07:22:34.707535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:33:19.436850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026801759999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20338,"num_read_kb":38377,"num_write":424,"num_write_kb":22354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"163'468","reported_seq":21373,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765348+0000","last_change":"2024-09-12T07:22:22.151697+0000","last_active":"2024-09-12T07:22:34.765348+0000","last_peered":"2024-09-12T07:22:34.765348+0000","last_clean":"2024-09-12T07:22:34.765348+0000","last_became_active":"2024-09-12T07:22:22.149491+0000","last_became_peered":"2024-09-12T07:22:22.149491+0000","last_unstale":"2024-09-12T07:22:34.765348+0000","last_undegraded":"2024-09-12T07:22:34.765348+0000","last_fullsized":"2024-09-12T07:22:34.765348+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:51.871543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010058409999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":20924,"num_read_kb":39045,"num_write":507,"num_write_kb":15255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":23048347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"163'607","reported_seq":3893,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764874+0000","last_change":"2024-09-12T07:22:21.150401+0000","last_active":"2024-09-12T07:22:34.764874+0000","last_peered":"2024-09-12T07:22:34.764874+0000","last_clean":"2024-09-12T07:22:34.764874+0000","last_became_active":"2024-09-12T07:22:21.148942+0000","last_became_peered":"2024-09-12T07:22:21.148942+0000","last_unstale":"2024-09-12T07:22:34.764874+0000","last_undegraded":"2024-09-12T07:22:34.764874+0000","last_fullsized":"2024-09-12T07:22:34.764874+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:45.508340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014959909999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":3878,"num_read_kb":17362,"num_write":791,"num_write_kb":11625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":8199265,"num_keys_recovered":174,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"162'447","reported_seq":22008,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264233+0000","last_change":"2024-09-12T07:22:06.195928+0000","last_active":"2024-09-12T07:22:34.264233+0000","last_peered":"2024-09-12T07:22:34.264233+0000","last_clean":"2024-09-12T07:22:34.264233+0000","last_became_active":"2024-09-12T07:22:01.145404+0000","last_became_peered":"2024-09-12T07:22:01.145404+0000","last_unstale":"2024-09-12T07:22:34.264233+0000","last_undegraded":"2024-09-12T07:22:34.264233+0000","last_fullsized":"2024-09-12T07:22:34.264233+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:03.385436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014847230000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":21866,"num_read_kb":61051,"num_write":426,"num_write_kb":15720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":12377120,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'388","reported_seq":4868,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249862+0000","last_change":"2024-09-12T07:22:27.215825+0000","last_active":"2024-09-12T07:22:34.249862+0000","last_peered":"2024-09-12T07:22:34.249862+0000","last_clean":"2024-09-12T07:22:34.249862+0000","last_became_active":"2024-09-12T07:22:27.213519+0000","last_became_peered":"2024-09-12T07:22:27.213519+0000","last_unstale":"2024-09-12T07:22:34.249862+0000","last_undegraded":"2024-09-12T07:22:34.249862+0000","last_fullsized":"2024-09-12T07:22:34.249862+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.924110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022138280000000001,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4657,"num_read_kb":13211,"num_write":331,"num_write_kb":17325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":27687211,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"163'649","reported_seq":7971,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764092+0000","last_change":"2024-09-12T07:22:21.149665+0000","last_active":"2024-09-12T07:22:34.764092+0000","last_peered":"2024-09-12T07:22:34.764092+0000","last_clean":"2024-09-12T07:22:34.764092+0000","last_became_active":"2024-09-12T07:22:21.147583+0000","last_became_peered":"2024-09-12T07:22:21.147583+0000","last_unstale":"2024-09-12T07:22:34.764092+0000","last_undegraded":"2024-09-12T07:22:34.764092+0000","last_fullsized":"2024-09-12T07:22:34.764092+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:22.335622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023936500000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8436,"num_read_kb":50796,"num_write":779,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":7815100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"163'609","reported_seq":3818,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707368+0000","last_change":"2024-09-12T07:22:06.173059+0000","last_active":"2024-09-12T07:22:34.707368+0000","last_peered":"2024-09-12T07:22:34.707368+0000","last_clean":"2024-09-12T07:22:34.707368+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:22:34.707368+0000","last_undegraded":"2024-09-12T07:22:34.707368+0000","last_fullsized":"2024-09-12T07:22:34.707368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:17.727732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099897500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6008,"num_read_kb":52191,"num_write":744,"num_write_kb":22227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"162'653","reported_seq":7128,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264260+0000","last_change":"2024-09-12T07:22:22.149915+0000","last_active":"2024-09-12T07:22:34.264260+0000","last_peered":"2024-09-12T07:22:34.264260+0000","last_clean":"2024-09-12T07:22:34.264260+0000","last_became_active":"2024-09-12T07:22:22.148392+0000","last_became_peered":"2024-09-12T07:22:22.148392+0000","last_unstale":"2024-09-12T07:22:34.264260+0000","last_undegraded":"2024-09-12T07:22:34.264260+0000","last_fullsized":"2024-09-12T07:22:34.264260+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:29.255739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002496505,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8906,"num_read_kb":19832,"num_write":605,"num_write_kb":22990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":20211745,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"163'375","reported_seq":1927,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765128+0000","last_change":"2024-09-12T07:22:21.149525+0000","last_active":"2024-09-12T07:22:34.765128+0000","last_peered":"2024-09-12T07:22:34.765128+0000","last_clean":"2024-09-12T07:22:34.765128+0000","last_became_active":"2024-09-12T07:22:21.148026+0000","last_became_peered":"2024-09-12T07:22:21.148026+0000","last_unstale":"2024-09-12T07:22:34.765128+0000","last_undegraded":"2024-09-12T07:22:34.765128+0000","last_fullsized":"2024-09-12T07:22:34.765128+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:15.887884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017138209999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1348,"num_read_kb":18539,"num_write":421,"num_write_kb":14413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":10121363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"152'344","reported_seq":7171,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765014+0000","last_change":"2024-09-12T07:22:06.164819+0000","last_active":"2024-09-12T07:22:34.765014+0000","last_peered":"2024-09-12T07:22:34.765014+0000","last_clean":"2024-09-12T07:22:34.765014+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:22:34.765014+0000","last_undegraded":"2024-09-12T07:22:34.765014+0000","last_fullsized":"2024-09-12T07:22:34.765014+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:45.370726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080767000000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7070,"num_read_kb":53310,"num_write":299,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"160'391","reported_seq":5850,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.250009+0000","last_change":"2024-09-12T07:22:27.214683+0000","last_active":"2024-09-12T07:22:34.250009+0000","last_peered":"2024-09-12T07:22:34.250009+0000","last_clean":"2024-09-12T07:22:34.250009+0000","last_became_active":"2024-09-12T07:22:27.212420+0000","last_became_peered":"2024-09-12T07:22:27.212420+0000","last_unstale":"2024-09-12T07:22:34.250009+0000","last_undegraded":"2024-09-12T07:22:34.250009+0000","last_fullsized":"2024-09-12T07:22:34.250009+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:16.234305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097451600000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5380,"num_read_kb":60804,"num_write":390,"num_write_kb":30252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":31149476,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764999+0000","last_change":"2024-09-12T07:22:31.308130+0000","last_active":"2024-09-12T07:22:34.764999+0000","last_peered":"2024-09-12T07:22:34.764999+0000","last_clean":"2024-09-12T07:22:34.764999+0000","last_became_active":"2024-09-12T07:22:31.307821+0000","last_became_peered":"2024-09-12T07:22:31.307821+0000","last_unstale":"2024-09-12T07:22:34.764999+0000","last_undegraded":"2024-09-12T07:22:34.764999+0000","last_fullsized":"2024-09-12T07:22:34.764999+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:45.276396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"163'639","reported_seq":6266,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264231+0000","last_change":"2024-09-12T07:22:21.153348+0000","last_active":"2024-09-12T07:22:34.264231+0000","last_peered":"2024-09-12T07:22:34.264231+0000","last_clean":"2024-09-12T07:22:34.264231+0000","last_became_active":"2024-09-12T07:22:21.152036+0000","last_became_peered":"2024-09-12T07:22:21.152036+0000","last_unstale":"2024-09-12T07:22:34.264231+0000","last_undegraded":"2024-09-12T07:22:34.264231+0000","last_fullsized":"2024-09-12T07:22:34.264231+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:16.901686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014037019999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6009,"num_read_kb":23012,"num_write":704,"num_write_kb":11415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":1480654,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764229+0000","last_change":"2024-09-12T07:22:18.173512+0000","last_active":"2024-09-12T07:22:34.764229+0000","last_peered":"2024-09-12T07:22:34.764229+0000","last_clean":"2024-09-12T07:22:34.764229+0000","last_became_active":"2024-09-12T07:22:18.173148+0000","last_became_peered":"2024-09-12T07:22:18.173148+0000","last_unstale":"2024-09-12T07:22:34.764229+0000","last_undegraded":"2024-09-12T07:22:34.764229+0000","last_fullsized":"2024-09-12T07:22:34.764229+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:07.135570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"162'679","reported_seq":2209,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707544+0000","last_change":"2024-09-12T07:22:06.174857+0000","last_active":"2024-09-12T07:22:34.707544+0000","last_peered":"2024-09-12T07:22:34.707544+0000","last_clean":"2024-09-12T07:22:34.707544+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:22:34.707544+0000","last_undegraded":"2024-09-12T07:22:34.707544+0000","last_fullsized":"2024-09-12T07:22:34.707544+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:16.561231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025712959999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3057,"num_read_kb":14289,"num_write":898,"num_write_kb":11329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":348,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764698+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:22:34.764698+0000","last_peered":"2024-09-12T07:22:34.764698+0000","last_clean":"2024-09-12T07:22:34.764698+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:22:34.764698+0000","last_undegraded":"2024-09-12T07:22:34.764698+0000","last_fullsized":"2024-09-12T07:22:34.764698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"166'506","reported_seq":3850,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764186+0000","last_change":"2024-09-12T07:22:21.149345+0000","last_active":"2024-09-12T07:22:34.764186+0000","last_peered":"2024-09-12T07:22:34.764186+0000","last_clean":"2024-09-12T07:22:34.764186+0000","last_became_active":"2024-09-12T07:22:21.147703+0000","last_became_peered":"2024-09-12T07:22:21.147703+0000","last_unstale":"2024-09-12T07:22:34.764186+0000","last_undegraded":"2024-09-12T07:22:34.764186+0000","last_fullsized":"2024-09-12T07:22:34.764186+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:05.437283+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.023667932999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4682,"num_read_kb":14844,"num_write":579,"num_write_kb":8813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6163290,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764154+0000","last_change":"2024-09-12T07:22:18.169631+0000","last_active":"2024-09-12T07:22:34.764154+0000","last_peered":"2024-09-12T07:22:34.764154+0000","last_clean":"2024-09-12T07:22:34.764154+0000","last_became_active":"2024-09-12T07:22:18.169285+0000","last_became_peered":"2024-09-12T07:22:18.169285+0000","last_unstale":"2024-09-12T07:22:34.764154+0000","last_undegraded":"2024-09-12T07:22:34.764154+0000","last_fullsized":"2024-09-12T07:22:34.764154+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:21.373190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"166'565","reported_seq":10840,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765459+0000","last_change":"2024-09-12T07:22:22.153784+0000","last_active":"2024-09-12T07:22:34.765459+0000","last_peered":"2024-09-12T07:22:34.765459+0000","last_clean":"2024-09-12T07:22:34.765459+0000","last_became_active":"2024-09-12T07:22:22.151818+0000","last_became_peered":"2024-09-12T07:22:22.151818+0000","last_unstale":"2024-09-12T07:22:34.765459+0000","last_undegraded":"2024-09-12T07:22:34.765459+0000","last_fullsized":"2024-09-12T07:22:34.765459+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:02.130727+0000","scrub_duration":0,"objects_trimmed":18,"snaptrim_duration":0.023276249999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11624,"num_read_kb":77562,"num_write":595,"num_write_kb":25411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":22089334,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"163'503","reported_seq":4495,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249801+0000","last_change":"2024-09-12T07:22:27.215301+0000","last_active":"2024-09-12T07:22:34.249801+0000","last_peered":"2024-09-12T07:22:34.249801+0000","last_clean":"2024-09-12T07:22:34.249801+0000","last_became_active":"2024-09-12T07:22:27.213073+0000","last_became_peered":"2024-09-12T07:22:27.213073+0000","last_unstale":"2024-09-12T07:22:34.249801+0000","last_undegraded":"2024-09-12T07:22:34.249801+0000","last_fullsized":"2024-09-12T07:22:34.249801+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:10.316262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018780750000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":3159,"num_read_kb":12201,"num_write":460,"num_write_kb":10579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":2603172,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":48,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765232+0000","last_change":"2024-09-12T07:22:19.200232+0000","last_active":"2024-09-12T07:22:34.765232+0000","last_peered":"2024-09-12T07:22:34.765232+0000","last_clean":"2024-09-12T07:22:34.765232+0000","last_became_active":"2024-09-12T07:22:19.178595+0000","last_became_peered":"2024-09-12T07:22:19.178595+0000","last_unstale":"2024-09-12T07:22:34.765232+0000","last_undegraded":"2024-09-12T07:22:34.765232+0000","last_fullsized":"2024-09-12T07:22:34.765232+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:58.776762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"166'781","reported_seq":3325,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764213+0000","last_change":"2024-09-12T07:22:21.150766+0000","last_active":"2024-09-12T07:22:34.764213+0000","last_peered":"2024-09-12T07:22:34.764213+0000","last_clean":"2024-09-12T07:22:34.764213+0000","last_became_active":"2024-09-12T07:22:21.147758+0000","last_became_peered":"2024-09-12T07:22:21.147758+0000","last_unstale":"2024-09-12T07:22:34.764213+0000","last_undegraded":"2024-09-12T07:22:34.764213+0000","last_fullsized":"2024-09-12T07:22:34.764213+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:15.197467+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.033858052999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5450,"num_read_kb":11647,"num_write":1052,"num_write_kb":13386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":9483314,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":299,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249699+0000","last_change":"2024-09-12T07:22:09.183569+0000","last_active":"2024-09-12T07:22:34.249699+0000","last_peered":"2024-09-12T07:22:34.249699+0000","last_clean":"2024-09-12T07:22:34.249699+0000","last_became_active":"2024-09-12T07:22:09.182547+0000","last_became_peered":"2024-09-12T07:22:09.182547+0000","last_unstale":"2024-09-12T07:22:34.249699+0000","last_undegraded":"2024-09-12T07:22:34.249699+0000","last_fullsized":"2024-09-12T07:22:34.249699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:52.520157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"162'406","reported_seq":10545,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264069+0000","last_change":"2024-09-12T07:22:06.195337+0000","last_active":"2024-09-12T07:22:34.264069+0000","last_peered":"2024-09-12T07:22:34.264069+0000","last_clean":"2024-09-12T07:22:34.264069+0000","last_became_active":"2024-09-12T07:22:01.146421+0000","last_became_peered":"2024-09-12T07:22:01.146421+0000","last_unstale":"2024-09-12T07:22:34.264069+0000","last_undegraded":"2024-09-12T07:22:34.264069+0000","last_fullsized":"2024-09-12T07:22:34.264069+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:38.812988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094442799999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11174,"num_read_kb":59795,"num_write":376,"num_write_kb":27716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":22543830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":360,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764760+0000","last_change":"2024-09-12T07:22:18.171696+0000","last_active":"2024-09-12T07:22:34.764760+0000","last_peered":"2024-09-12T07:22:34.764760+0000","last_clean":"2024-09-12T07:22:34.764760+0000","last_became_active":"2024-09-12T07:22:18.171266+0000","last_became_peered":"2024-09-12T07:22:18.171266+0000","last_unstale":"2024-09-12T07:22:34.764760+0000","last_undegraded":"2024-09-12T07:22:34.764760+0000","last_fullsized":"2024-09-12T07:22:34.764760+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:26.361507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"162'1062","reported_seq":6945,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707508+0000","last_change":"2024-09-12T07:22:22.152024+0000","last_active":"2024-09-12T07:22:34.707508+0000","last_peered":"2024-09-12T07:22:34.707508+0000","last_clean":"2024-09-12T07:22:34.707508+0000","last_became_active":"2024-09-12T07:22:22.150316+0000","last_became_peered":"2024-09-12T07:22:22.150316+0000","last_unstale":"2024-09-12T07:22:34.707508+0000","last_undegraded":"2024-09-12T07:22:34.707508+0000","last_fullsized":"2024-09-12T07:22:34.707508+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:57:13.384646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021189680000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8933,"num_read_kb":59814,"num_write":1570,"num_write_kb":27462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":14379761,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":45,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764860+0000","last_change":"2024-09-12T07:22:19.178038+0000","last_active":"2024-09-12T07:22:34.764860+0000","last_peered":"2024-09-12T07:22:34.764860+0000","last_clean":"2024-09-12T07:22:34.764860+0000","last_became_active":"2024-09-12T07:22:19.170715+0000","last_became_peered":"2024-09-12T07:22:19.170715+0000","last_unstale":"2024-09-12T07:22:34.764860+0000","last_undegraded":"2024-09-12T07:22:34.764860+0000","last_fullsized":"2024-09-12T07:22:34.764860+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:28.982893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":439,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764900+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:22:34.764900+0000","last_peered":"2024-09-12T07:22:34.764900+0000","last_clean":"2024-09-12T07:22:34.764900+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:22:34.764900+0000","last_undegraded":"2024-09-12T07:22:34.764900+0000","last_fullsized":"2024-09-12T07:22:34.764900+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"162'962","reported_seq":3107,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707372+0000","last_change":"2024-09-12T07:22:21.151517+0000","last_active":"2024-09-12T07:22:34.707372+0000","last_peered":"2024-09-12T07:22:34.707372+0000","last_clean":"2024-09-12T07:22:34.707372+0000","last_became_active":"2024-09-12T07:22:21.150083+0000","last_became_peered":"2024-09-12T07:22:21.150083+0000","last_unstale":"2024-09-12T07:22:34.707372+0000","last_undegraded":"2024-09-12T07:22:34.707372+0000","last_fullsized":"2024-09-12T07:22:34.707372+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":962,"log_dups_size":0,"ondisk_log_size":962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:38.471826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011038020000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5044,"num_read_kb":23221,"num_write":1448,"num_write_kb":24489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":21462395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"163'549","reported_seq":5426,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249824+0000","last_change":"2024-09-12T07:22:27.215048+0000","last_active":"2024-09-12T07:22:34.249824+0000","last_peered":"2024-09-12T07:22:34.249824+0000","last_clean":"2024-09-12T07:22:34.249824+0000","last_became_active":"2024-09-12T07:22:27.212942+0000","last_became_peered":"2024-09-12T07:22:27.212942+0000","last_unstale":"2024-09-12T07:22:34.249824+0000","last_undegraded":"2024-09-12T07:22:34.249824+0000","last_fullsized":"2024-09-12T07:22:34.249824+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:59.386875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021244950000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7563,"num_read_kb":32357,"num_write":525,"num_write_kb":20455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1710555,"num_keys_recovered":41,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"162'540","reported_seq":6677,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249766+0000","last_change":"2024-09-12T07:22:27.215838+0000","last_active":"2024-09-12T07:22:34.249766+0000","last_peered":"2024-09-12T07:22:34.249766+0000","last_clean":"2024-09-12T07:22:34.249766+0000","last_became_active":"2024-09-12T07:22:27.213533+0000","last_became_peered":"2024-09-12T07:22:27.213533+0000","last_unstale":"2024-09-12T07:22:34.249766+0000","last_undegraded":"2024-09-12T07:22:34.249766+0000","last_fullsized":"2024-09-12T07:22:34.249766+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:38.800561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069883599999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":6247,"num_read_kb":49000,"num_write":522,"num_write_kb":25535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":42565291,"num_keys_recovered":104,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"162'692","reported_seq":3567,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707411+0000","last_change":"2024-09-12T07:22:06.173724+0000","last_active":"2024-09-12T07:22:34.707411+0000","last_peered":"2024-09-12T07:22:34.707411+0000","last_clean":"2024-09-12T07:22:34.707411+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:22:34.707411+0000","last_undegraded":"2024-09-12T07:22:34.707411+0000","last_fullsized":"2024-09-12T07:22:34.707411+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:30.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015261610000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6009,"num_read_kb":21069,"num_write":727,"num_write_kb":17071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"162'445","reported_seq":9922,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764933+0000","last_change":"2024-09-12T07:22:22.151400+0000","last_active":"2024-09-12T07:22:34.764933+0000","last_peered":"2024-09-12T07:22:34.764933+0000","last_clean":"2024-09-12T07:22:34.764933+0000","last_became_active":"2024-09-12T07:22:22.149715+0000","last_became_peered":"2024-09-12T07:22:22.149715+0000","last_unstale":"2024-09-12T07:22:34.764933+0000","last_undegraded":"2024-09-12T07:22:34.764933+0000","last_fullsized":"2024-09-12T07:22:34.764933+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:51.662782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016506240000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11019,"num_read_kb":49685,"num_write":382,"num_write_kb":15924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":18973449,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"161'769","reported_seq":3640,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765503+0000","last_change":"2024-09-12T07:22:22.153946+0000","last_active":"2024-09-12T07:22:34.765503+0000","last_peered":"2024-09-12T07:22:34.765503+0000","last_clean":"2024-09-12T07:22:34.765503+0000","last_became_active":"2024-09-12T07:22:22.152455+0000","last_became_peered":"2024-09-12T07:22:22.152455+0000","last_unstale":"2024-09-12T07:22:34.765503+0000","last_undegraded":"2024-09-12T07:22:34.765503+0000","last_fullsized":"2024-09-12T07:22:34.765503+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:17.209337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014515540000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5118,"num_read_kb":30709,"num_write":1176,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":7370075,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"162'531","reported_seq":5409,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765193+0000","last_change":"2024-09-12T07:22:06.166758+0000","last_active":"2024-09-12T07:22:34.765193+0000","last_peered":"2024-09-12T07:22:34.765193+0000","last_clean":"2024-09-12T07:22:34.765193+0000","last_became_active":"2024-09-12T07:22:00.145108+0000","last_became_peered":"2024-09-12T07:22:00.145108+0000","last_unstale":"2024-09-12T07:22:34.765193+0000","last_undegraded":"2024-09-12T07:22:34.765193+0000","last_fullsized":"2024-09-12T07:22:34.765193+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:31.569737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801299999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6080,"num_read_kb":38867,"num_write":581,"num_write_kb":16891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":10508628,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"162'596","reported_seq":3023,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765441+0000","last_change":"2024-09-12T07:22:06.166247+0000","last_active":"2024-09-12T07:22:34.765441+0000","last_peered":"2024-09-12T07:22:34.765441+0000","last_clean":"2024-09-12T07:22:34.765441+0000","last_became_active":"2024-09-12T07:22:01.143359+0000","last_became_peered":"2024-09-12T07:22:01.143359+0000","last_unstale":"2024-09-12T07:22:34.765441+0000","last_undegraded":"2024-09-12T07:22:34.765441+0000","last_fullsized":"2024-09-12T07:22:34.765441+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:49.338904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001515553,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3463,"num_read_kb":13505,"num_write":637,"num_write_kb":16522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":8725508,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"152'338","reported_seq":1797,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765144+0000","last_change":"2024-09-12T07:22:31.247099+0000","last_active":"2024-09-12T07:22:34.765144+0000","last_peered":"2024-09-12T07:22:34.765144+0000","last_clean":"2024-09-12T07:22:34.765144+0000","last_became_active":"2024-09-12T07:22:31.245905+0000","last_became_peered":"2024-09-12T07:22:31.245905+0000","last_unstale":"2024-09-12T07:22:34.765144+0000","last_undegraded":"2024-09-12T07:22:34.765144+0000","last_fullsized":"2024-09-12T07:22:34.765144+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:10.350713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816669999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1472,"num_read_kb":27101,"num_write":327,"num_write_kb":18963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"163'454","reported_seq":7380,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764987+0000","last_change":"2024-09-12T07:22:06.164601+0000","last_active":"2024-09-12T07:22:34.764987+0000","last_peered":"2024-09-12T07:22:34.764987+0000","last_clean":"2024-09-12T07:22:34.764987+0000","last_became_active":"2024-09-12T07:22:01.143631+0000","last_became_peered":"2024-09-12T07:22:01.143631+0000","last_unstale":"2024-09-12T07:22:34.764987+0000","last_undegraded":"2024-09-12T07:22:34.764987+0000","last_fullsized":"2024-09-12T07:22:34.764987+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:34.629049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065451099999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7090,"num_read_kb":47369,"num_write":461,"num_write_kb":21538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":17766842,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"159'408","reported_seq":11210,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264074+0000","last_change":"2024-09-12T07:22:06.196502+0000","last_active":"2024-09-12T07:22:34.264074+0000","last_peered":"2024-09-12T07:22:34.264074+0000","last_clean":"2024-09-12T07:22:34.264074+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:22:34.264074+0000","last_undegraded":"2024-09-12T07:22:34.264074+0000","last_fullsized":"2024-09-12T07:22:34.264074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:48.940535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019415999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11863,"num_read_kb":47954,"num_write":410,"num_write_kb":15586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'476","reported_seq":19728,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765222+0000","last_change":"2024-09-12T07:22:06.165173+0000","last_active":"2024-09-12T07:22:34.765222+0000","last_peered":"2024-09-12T07:22:34.765222+0000","last_clean":"2024-09-12T07:22:34.765222+0000","last_became_active":"2024-09-12T07:22:01.143577+0000","last_became_peered":"2024-09-12T07:22:01.143577+0000","last_unstale":"2024-09-12T07:22:34.765222+0000","last_undegraded":"2024-09-12T07:22:34.765222+0000","last_fullsized":"2024-09-12T07:22:34.765222+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:12:53.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010515870000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":18947,"num_read_kb":82796,"num_write":553,"num_write_kb":44274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":53090062,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":323,"num_object_clones":0,"num_object_copies":646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":319,"num_read":260585,"num_read_kb":1190964,"num_write":20483,"num_write_kb":597347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5845,"num_bytes_recovered":432794818,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},"log_size":17959,"ondisk_log_size":17959,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56164,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315676,"statfs":{"total":96636764160,"available":96579252224,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33376,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338464,"statfs":{"total":96636764160,"available":96602587136,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55576,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316264,"statfs":{"total":96636764160,"available":96579854336,"internally_reserved":0,"allocated":2232320,"data_stored":1075567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51432,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320408,"statfs":{"total":96636764160,"available":96584097792,"internally_reserved":0,"allocated":2244608,"data_stored":1081409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69480,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302360,"statfs":{"total":96636764160,"available":96565616640,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":165,"seq":708669603848,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31008,"kb_used_data":2188,"kb_used_omap":23,"kb_used_meta":28776,"kb_avail":94340832,"statfs":{"total":96636764160,"available":96605011968,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24213,"internal_metadata":29466987},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55592,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316248,"statfs":{"total":96636764160,"available":96579837952,"internally_reserved":0,"allocated":2248704,"data_stored":1081428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64488,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307352,"statfs":{"total":96636764160,"available":96570728448,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:22:39.122 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T07:22:39.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:22:39.423 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:22:39.423 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:22:39.435 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-12T07:22:37.525611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":327,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":319,"num_read":260651,"num_read_kb":1191019,"num_write":20586,"num_write_kb":598977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5849,"num_bytes_recovered":432794875,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18034,"ondisk_log_size":18034,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417116,"kb_used_data":18668,"kb_used_omap":34,"kb_used_meta":398237,"kb_avail":754557604,"statfs":{"total":773094113280,"available":772666986496,"internally_reserved":0,"allocated":19116032,"data_stored":9826090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35370,"internal_metadata":407795158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001526"},"pg_stats":[{"pgid":"3.1f","version":"166'638","reported_seq":6785,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264251+0000","last_change":"2024-09-12T07:22:22.149919+0000","last_active":"2024-09-12T07:22:34.264251+0000","last_peered":"2024-09-12T07:22:34.264251+0000","last_clean":"2024-09-12T07:22:34.264251+0000","last_became_active":"2024-09-12T07:22:22.148401+0000","last_became_peered":"2024-09-12T07:22:22.148401+0000","last_unstale":"2024-09-12T07:22:34.264251+0000","last_undegraded":"2024-09-12T07:22:34.264251+0000","last_fullsized":"2024-09-12T07:22:34.264251+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:27:39.529993+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.029437784000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7771,"num_read_kb":29649,"num_write":783,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":2971697,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"161'489","reported_seq":2632,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707535+0000","last_change":"2024-09-12T07:22:06.175231+0000","last_active":"2024-09-12T07:22:34.707535+0000","last_peered":"2024-09-12T07:22:34.707535+0000","last_clean":"2024-09-12T07:22:34.707535+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:22:34.707535+0000","last_undegraded":"2024-09-12T07:22:34.707535+0000","last_fullsized":"2024-09-12T07:22:34.707535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:33:19.436850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026801759999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20338,"num_read_kb":38377,"num_write":424,"num_write_kb":22354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"163'468","reported_seq":21373,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765348+0000","last_change":"2024-09-12T07:22:22.151697+0000","last_active":"2024-09-12T07:22:34.765348+0000","last_peered":"2024-09-12T07:22:34.765348+0000","last_clean":"2024-09-12T07:22:34.765348+0000","last_became_active":"2024-09-12T07:22:22.149491+0000","last_became_peered":"2024-09-12T07:22:22.149491+0000","last_unstale":"2024-09-12T07:22:34.765348+0000","last_undegraded":"2024-09-12T07:22:34.765348+0000","last_fullsized":"2024-09-12T07:22:34.765348+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:51.871543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010058409999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":20924,"num_read_kb":39045,"num_write":507,"num_write_kb":15255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":23048347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"163'607","reported_seq":3893,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764874+0000","last_change":"2024-09-12T07:22:21.150401+0000","last_active":"2024-09-12T07:22:34.764874+0000","last_peered":"2024-09-12T07:22:34.764874+0000","last_clean":"2024-09-12T07:22:34.764874+0000","last_became_active":"2024-09-12T07:22:21.148942+0000","last_became_peered":"2024-09-12T07:22:21.148942+0000","last_unstale":"2024-09-12T07:22:34.764874+0000","last_undegraded":"2024-09-12T07:22:34.764874+0000","last_fullsized":"2024-09-12T07:22:34.764874+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:45.508340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014959909999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":3878,"num_read_kb":17362,"num_write":791,"num_write_kb":11625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":8199265,"num_keys_recovered":174,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"162'447","reported_seq":22008,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264233+0000","last_change":"2024-09-12T07:22:06.195928+0000","last_active":"2024-09-12T07:22:34.264233+0000","last_peered":"2024-09-12T07:22:34.264233+0000","last_clean":"2024-09-12T07:22:34.264233+0000","last_became_active":"2024-09-12T07:22:01.145404+0000","last_became_peered":"2024-09-12T07:22:01.145404+0000","last_unstale":"2024-09-12T07:22:34.264233+0000","last_undegraded":"2024-09-12T07:22:34.264233+0000","last_fullsized":"2024-09-12T07:22:34.264233+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:03.385436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014847230000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":21866,"num_read_kb":61051,"num_write":426,"num_write_kb":15720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":12377120,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'388","reported_seq":4868,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249862+0000","last_change":"2024-09-12T07:22:27.215825+0000","last_active":"2024-09-12T07:22:34.249862+0000","last_peered":"2024-09-12T07:22:34.249862+0000","last_clean":"2024-09-12T07:22:34.249862+0000","last_became_active":"2024-09-12T07:22:27.213519+0000","last_became_peered":"2024-09-12T07:22:27.213519+0000","last_unstale":"2024-09-12T07:22:34.249862+0000","last_undegraded":"2024-09-12T07:22:34.249862+0000","last_fullsized":"2024-09-12T07:22:34.249862+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.924110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022138280000000001,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4657,"num_read_kb":13211,"num_write":331,"num_write_kb":17325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":27687211,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"163'649","reported_seq":7971,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764092+0000","last_change":"2024-09-12T07:22:21.149665+0000","last_active":"2024-09-12T07:22:34.764092+0000","last_peered":"2024-09-12T07:22:34.764092+0000","last_clean":"2024-09-12T07:22:34.764092+0000","last_became_active":"2024-09-12T07:22:21.147583+0000","last_became_peered":"2024-09-12T07:22:21.147583+0000","last_unstale":"2024-09-12T07:22:34.764092+0000","last_undegraded":"2024-09-12T07:22:34.764092+0000","last_fullsized":"2024-09-12T07:22:34.764092+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:22.335622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023936500000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8436,"num_read_kb":50796,"num_write":779,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":7815100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"163'609","reported_seq":3818,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707368+0000","last_change":"2024-09-12T07:22:06.173059+0000","last_active":"2024-09-12T07:22:34.707368+0000","last_peered":"2024-09-12T07:22:34.707368+0000","last_clean":"2024-09-12T07:22:34.707368+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:22:34.707368+0000","last_undegraded":"2024-09-12T07:22:34.707368+0000","last_fullsized":"2024-09-12T07:22:34.707368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:17.727732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099897500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6008,"num_read_kb":52191,"num_write":744,"num_write_kb":22227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"162'653","reported_seq":7128,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264260+0000","last_change":"2024-09-12T07:22:22.149915+0000","last_active":"2024-09-12T07:22:34.264260+0000","last_peered":"2024-09-12T07:22:34.264260+0000","last_clean":"2024-09-12T07:22:34.264260+0000","last_became_active":"2024-09-12T07:22:22.148392+0000","last_became_peered":"2024-09-12T07:22:22.148392+0000","last_unstale":"2024-09-12T07:22:34.264260+0000","last_undegraded":"2024-09-12T07:22:34.264260+0000","last_fullsized":"2024-09-12T07:22:34.264260+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:29.255739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002496505,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8906,"num_read_kb":19832,"num_write":605,"num_write_kb":22990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":20211745,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"163'375","reported_seq":1927,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765128+0000","last_change":"2024-09-12T07:22:21.149525+0000","last_active":"2024-09-12T07:22:34.765128+0000","last_peered":"2024-09-12T07:22:34.765128+0000","last_clean":"2024-09-12T07:22:34.765128+0000","last_became_active":"2024-09-12T07:22:21.148026+0000","last_became_peered":"2024-09-12T07:22:21.148026+0000","last_unstale":"2024-09-12T07:22:34.765128+0000","last_undegraded":"2024-09-12T07:22:34.765128+0000","last_fullsized":"2024-09-12T07:22:34.765128+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:15.887884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017138209999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1348,"num_read_kb":18539,"num_write":421,"num_write_kb":14413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":10121363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"152'344","reported_seq":7171,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765014+0000","last_change":"2024-09-12T07:22:06.164819+0000","last_active":"2024-09-12T07:22:34.765014+0000","last_peered":"2024-09-12T07:22:34.765014+0000","last_clean":"2024-09-12T07:22:34.765014+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:22:34.765014+0000","last_undegraded":"2024-09-12T07:22:34.765014+0000","last_fullsized":"2024-09-12T07:22:34.765014+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:45.370726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080767000000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7070,"num_read_kb":53310,"num_write":299,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"160'391","reported_seq":5850,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.250009+0000","last_change":"2024-09-12T07:22:27.214683+0000","last_active":"2024-09-12T07:22:34.250009+0000","last_peered":"2024-09-12T07:22:34.250009+0000","last_clean":"2024-09-12T07:22:34.250009+0000","last_became_active":"2024-09-12T07:22:27.212420+0000","last_became_peered":"2024-09-12T07:22:27.212420+0000","last_unstale":"2024-09-12T07:22:34.250009+0000","last_undegraded":"2024-09-12T07:22:34.250009+0000","last_fullsized":"2024-09-12T07:22:34.250009+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:16.234305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097451600000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5380,"num_read_kb":60804,"num_write":390,"num_write_kb":30252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":31149476,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764999+0000","last_change":"2024-09-12T07:22:31.308130+0000","last_active":"2024-09-12T07:22:34.764999+0000","last_peered":"2024-09-12T07:22:34.764999+0000","last_clean":"2024-09-12T07:22:34.764999+0000","last_became_active":"2024-09-12T07:22:31.307821+0000","last_became_peered":"2024-09-12T07:22:31.307821+0000","last_unstale":"2024-09-12T07:22:34.764999+0000","last_undegraded":"2024-09-12T07:22:34.764999+0000","last_fullsized":"2024-09-12T07:22:34.764999+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:45.276396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"163'639","reported_seq":6266,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264231+0000","last_change":"2024-09-12T07:22:21.153348+0000","last_active":"2024-09-12T07:22:34.264231+0000","last_peered":"2024-09-12T07:22:34.264231+0000","last_clean":"2024-09-12T07:22:34.264231+0000","last_became_active":"2024-09-12T07:22:21.152036+0000","last_became_peered":"2024-09-12T07:22:21.152036+0000","last_unstale":"2024-09-12T07:22:34.264231+0000","last_undegraded":"2024-09-12T07:22:34.264231+0000","last_fullsized":"2024-09-12T07:22:34.264231+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:16.901686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014037019999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6009,"num_read_kb":23012,"num_write":704,"num_write_kb":11415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":1480654,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764229+0000","last_change":"2024-09-12T07:22:18.173512+0000","last_active":"2024-09-12T07:22:34.764229+0000","last_peered":"2024-09-12T07:22:34.764229+0000","last_clean":"2024-09-12T07:22:34.764229+0000","last_became_active":"2024-09-12T07:22:18.173148+0000","last_became_peered":"2024-09-12T07:22:18.173148+0000","last_unstale":"2024-09-12T07:22:34.764229+0000","last_undegraded":"2024-09-12T07:22:34.764229+0000","last_fullsized":"2024-09-12T07:22:34.764229+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:07.135570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"162'679","reported_seq":2209,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707544+0000","last_change":"2024-09-12T07:22:06.174857+0000","last_active":"2024-09-12T07:22:34.707544+0000","last_peered":"2024-09-12T07:22:34.707544+0000","last_clean":"2024-09-12T07:22:34.707544+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:22:34.707544+0000","last_undegraded":"2024-09-12T07:22:34.707544+0000","last_fullsized":"2024-09-12T07:22:34.707544+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:16.561231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025712959999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3057,"num_read_kb":14289,"num_write":898,"num_write_kb":11329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":348,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764698+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:22:34.764698+0000","last_peered":"2024-09-12T07:22:34.764698+0000","last_clean":"2024-09-12T07:22:34.764698+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:22:34.764698+0000","last_undegraded":"2024-09-12T07:22:34.764698+0000","last_fullsized":"2024-09-12T07:22:34.764698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"166'506","reported_seq":3850,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764186+0000","last_change":"2024-09-12T07:22:21.149345+0000","last_active":"2024-09-12T07:22:34.764186+0000","last_peered":"2024-09-12T07:22:34.764186+0000","last_clean":"2024-09-12T07:22:34.764186+0000","last_became_active":"2024-09-12T07:22:21.147703+0000","last_became_peered":"2024-09-12T07:22:21.147703+0000","last_unstale":"2024-09-12T07:22:34.764186+0000","last_undegraded":"2024-09-12T07:22:34.764186+0000","last_fullsized":"2024-09-12T07:22:34.764186+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:05.437283+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.023667932999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4682,"num_read_kb":14844,"num_write":579,"num_write_kb":8813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6163290,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764154+0000","last_change":"2024-09-12T07:22:18.169631+0000","last_active":"2024-09-12T07:22:34.764154+0000","last_peered":"2024-09-12T07:22:34.764154+0000","last_clean":"2024-09-12T07:22:34.764154+0000","last_became_active":"2024-09-12T07:22:18.169285+0000","last_became_peered":"2024-09-12T07:22:18.169285+0000","last_unstale":"2024-09-12T07:22:34.764154+0000","last_undegraded":"2024-09-12T07:22:34.764154+0000","last_fullsized":"2024-09-12T07:22:34.764154+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:21.373190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"166'565","reported_seq":10840,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765459+0000","last_change":"2024-09-12T07:22:22.153784+0000","last_active":"2024-09-12T07:22:34.765459+0000","last_peered":"2024-09-12T07:22:34.765459+0000","last_clean":"2024-09-12T07:22:34.765459+0000","last_became_active":"2024-09-12T07:22:22.151818+0000","last_became_peered":"2024-09-12T07:22:22.151818+0000","last_unstale":"2024-09-12T07:22:34.765459+0000","last_undegraded":"2024-09-12T07:22:34.765459+0000","last_fullsized":"2024-09-12T07:22:34.765459+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:02.130727+0000","scrub_duration":0,"objects_trimmed":18,"snaptrim_duration":0.023276249999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11624,"num_read_kb":77562,"num_write":595,"num_write_kb":25411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":22089334,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"163'503","reported_seq":4495,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249801+0000","last_change":"2024-09-12T07:22:27.215301+0000","last_active":"2024-09-12T07:22:34.249801+0000","last_peered":"2024-09-12T07:22:34.249801+0000","last_clean":"2024-09-12T07:22:34.249801+0000","last_became_active":"2024-09-12T07:22:27.213073+0000","last_became_peered":"2024-09-12T07:22:27.213073+0000","last_unstale":"2024-09-12T07:22:34.249801+0000","last_undegraded":"2024-09-12T07:22:34.249801+0000","last_fullsized":"2024-09-12T07:22:34.249801+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:10.316262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018780750000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":3159,"num_read_kb":12201,"num_write":460,"num_write_kb":10579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":2603172,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":48,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765232+0000","last_change":"2024-09-12T07:22:19.200232+0000","last_active":"2024-09-12T07:22:34.765232+0000","last_peered":"2024-09-12T07:22:34.765232+0000","last_clean":"2024-09-12T07:22:34.765232+0000","last_became_active":"2024-09-12T07:22:19.178595+0000","last_became_peered":"2024-09-12T07:22:19.178595+0000","last_unstale":"2024-09-12T07:22:34.765232+0000","last_undegraded":"2024-09-12T07:22:34.765232+0000","last_fullsized":"2024-09-12T07:22:34.765232+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:58.776762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"166'781","reported_seq":3325,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764213+0000","last_change":"2024-09-12T07:22:21.150766+0000","last_active":"2024-09-12T07:22:34.764213+0000","last_peered":"2024-09-12T07:22:34.764213+0000","last_clean":"2024-09-12T07:22:34.764213+0000","last_became_active":"2024-09-12T07:22:21.147758+0000","last_became_peered":"2024-09-12T07:22:21.147758+0000","last_unstale":"2024-09-12T07:22:34.764213+0000","last_undegraded":"2024-09-12T07:22:34.764213+0000","last_fullsized":"2024-09-12T07:22:34.764213+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:15.197467+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.033858052999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5450,"num_read_kb":11647,"num_write":1052,"num_write_kb":13386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":9483314,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":299,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249699+0000","last_change":"2024-09-12T07:22:09.183569+0000","last_active":"2024-09-12T07:22:34.249699+0000","last_peered":"2024-09-12T07:22:34.249699+0000","last_clean":"2024-09-12T07:22:34.249699+0000","last_became_active":"2024-09-12T07:22:09.182547+0000","last_became_peered":"2024-09-12T07:22:09.182547+0000","last_unstale":"2024-09-12T07:22:34.249699+0000","last_undegraded":"2024-09-12T07:22:34.249699+0000","last_fullsized":"2024-09-12T07:22:34.249699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:52.520157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"162'406","reported_seq":10545,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264069+0000","last_change":"2024-09-12T07:22:06.195337+0000","last_active":"2024-09-12T07:22:34.264069+0000","last_peered":"2024-09-12T07:22:34.264069+0000","last_clean":"2024-09-12T07:22:34.264069+0000","last_became_active":"2024-09-12T07:22:01.146421+0000","last_became_peered":"2024-09-12T07:22:01.146421+0000","last_unstale":"2024-09-12T07:22:34.264069+0000","last_undegraded":"2024-09-12T07:22:34.264069+0000","last_fullsized":"2024-09-12T07:22:34.264069+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:38.812988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094442799999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11174,"num_read_kb":59795,"num_write":376,"num_write_kb":27716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":22543830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":360,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764760+0000","last_change":"2024-09-12T07:22:18.171696+0000","last_active":"2024-09-12T07:22:34.764760+0000","last_peered":"2024-09-12T07:22:34.764760+0000","last_clean":"2024-09-12T07:22:34.764760+0000","last_became_active":"2024-09-12T07:22:18.171266+0000","last_became_peered":"2024-09-12T07:22:18.171266+0000","last_unstale":"2024-09-12T07:22:34.764760+0000","last_undegraded":"2024-09-12T07:22:34.764760+0000","last_fullsized":"2024-09-12T07:22:34.764760+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:26.361507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"162'1062","reported_seq":6945,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707508+0000","last_change":"2024-09-12T07:22:22.152024+0000","last_active":"2024-09-12T07:22:34.707508+0000","last_peered":"2024-09-12T07:22:34.707508+0000","last_clean":"2024-09-12T07:22:34.707508+0000","last_became_active":"2024-09-12T07:22:22.150316+0000","last_became_peered":"2024-09-12T07:22:22.150316+0000","last_unstale":"2024-09-12T07:22:34.707508+0000","last_undegraded":"2024-09-12T07:22:34.707508+0000","last_fullsized":"2024-09-12T07:22:34.707508+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:57:13.384646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021189680000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8933,"num_read_kb":59814,"num_write":1570,"num_write_kb":27462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":14379761,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":45,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764860+0000","last_change":"2024-09-12T07:22:19.178038+0000","last_active":"2024-09-12T07:22:34.764860+0000","last_peered":"2024-09-12T07:22:34.764860+0000","last_clean":"2024-09-12T07:22:34.764860+0000","last_became_active":"2024-09-12T07:22:19.170715+0000","last_became_peered":"2024-09-12T07:22:19.170715+0000","last_unstale":"2024-09-12T07:22:34.764860+0000","last_undegraded":"2024-09-12T07:22:34.764860+0000","last_fullsized":"2024-09-12T07:22:34.764860+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:28.982893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":439,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764900+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:22:34.764900+0000","last_peered":"2024-09-12T07:22:34.764900+0000","last_clean":"2024-09-12T07:22:34.764900+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:22:34.764900+0000","last_undegraded":"2024-09-12T07:22:34.764900+0000","last_fullsized":"2024-09-12T07:22:34.764900+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"162'962","reported_seq":3107,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707372+0000","last_change":"2024-09-12T07:22:21.151517+0000","last_active":"2024-09-12T07:22:34.707372+0000","last_peered":"2024-09-12T07:22:34.707372+0000","last_clean":"2024-09-12T07:22:34.707372+0000","last_became_active":"2024-09-12T07:22:21.150083+0000","last_became_peered":"2024-09-12T07:22:21.150083+0000","last_unstale":"2024-09-12T07:22:34.707372+0000","last_undegraded":"2024-09-12T07:22:34.707372+0000","last_fullsized":"2024-09-12T07:22:34.707372+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":962,"log_dups_size":0,"ondisk_log_size":962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:38.471826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011038020000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5044,"num_read_kb":23221,"num_write":1448,"num_write_kb":24489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":21462395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"163'549","reported_seq":5426,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249824+0000","last_change":"2024-09-12T07:22:27.215048+0000","last_active":"2024-09-12T07:22:34.249824+0000","last_peered":"2024-09-12T07:22:34.249824+0000","last_clean":"2024-09-12T07:22:34.249824+0000","last_became_active":"2024-09-12T07:22:27.212942+0000","last_became_peered":"2024-09-12T07:22:27.212942+0000","last_unstale":"2024-09-12T07:22:34.249824+0000","last_undegraded":"2024-09-12T07:22:34.249824+0000","last_fullsized":"2024-09-12T07:22:34.249824+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:59.386875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021244950000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7563,"num_read_kb":32357,"num_write":525,"num_write_kb":20455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1710555,"num_keys_recovered":41,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"162'540","reported_seq":6677,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249766+0000","last_change":"2024-09-12T07:22:27.215838+0000","last_active":"2024-09-12T07:22:34.249766+0000","last_peered":"2024-09-12T07:22:34.249766+0000","last_clean":"2024-09-12T07:22:34.249766+0000","last_became_active":"2024-09-12T07:22:27.213533+0000","last_became_peered":"2024-09-12T07:22:27.213533+0000","last_unstale":"2024-09-12T07:22:34.249766+0000","last_undegraded":"2024-09-12T07:22:34.249766+0000","last_fullsized":"2024-09-12T07:22:34.249766+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:38.800561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069883599999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":6247,"num_read_kb":49000,"num_write":522,"num_write_kb":25535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":42565291,"num_keys_recovered":104,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"162'692","reported_seq":3567,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707411+0000","last_change":"2024-09-12T07:22:06.173724+0000","last_active":"2024-09-12T07:22:34.707411+0000","last_peered":"2024-09-12T07:22:34.707411+0000","last_clean":"2024-09-12T07:22:34.707411+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:22:34.707411+0000","last_undegraded":"2024-09-12T07:22:34.707411+0000","last_fullsized":"2024-09-12T07:22:34.707411+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:30.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015261610000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6009,"num_read_kb":21069,"num_write":727,"num_write_kb":17071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"162'445","reported_seq":9922,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764933+0000","last_change":"2024-09-12T07:22:22.151400+0000","last_active":"2024-09-12T07:22:34.764933+0000","last_peered":"2024-09-12T07:22:34.764933+0000","last_clean":"2024-09-12T07:22:34.764933+0000","last_became_active":"2024-09-12T07:22:22.149715+0000","last_became_peered":"2024-09-12T07:22:22.149715+0000","last_unstale":"2024-09-12T07:22:34.764933+0000","last_undegraded":"2024-09-12T07:22:34.764933+0000","last_fullsized":"2024-09-12T07:22:34.764933+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:51.662782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016506240000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11019,"num_read_kb":49685,"num_write":382,"num_write_kb":15924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":18973449,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"161'769","reported_seq":3640,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765503+0000","last_change":"2024-09-12T07:22:22.153946+0000","last_active":"2024-09-12T07:22:34.765503+0000","last_peered":"2024-09-12T07:22:34.765503+0000","last_clean":"2024-09-12T07:22:34.765503+0000","last_became_active":"2024-09-12T07:22:22.152455+0000","last_became_peered":"2024-09-12T07:22:22.152455+0000","last_unstale":"2024-09-12T07:22:34.765503+0000","last_undegraded":"2024-09-12T07:22:34.765503+0000","last_fullsized":"2024-09-12T07:22:34.765503+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:17.209337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014515540000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5118,"num_read_kb":30709,"num_write":1176,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":7370075,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"162'531","reported_seq":5409,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765193+0000","last_change":"2024-09-12T07:22:06.166758+0000","last_active":"2024-09-12T07:22:34.765193+0000","last_peered":"2024-09-12T07:22:34.765193+0000","last_clean":"2024-09-12T07:22:34.765193+0000","last_became_active":"2024-09-12T07:22:00.145108+0000","last_became_peered":"2024-09-12T07:22:00.145108+0000","last_unstale":"2024-09-12T07:22:34.765193+0000","last_undegraded":"2024-09-12T07:22:34.765193+0000","last_fullsized":"2024-09-12T07:22:34.765193+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:31.569737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801299999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6080,"num_read_kb":38867,"num_write":581,"num_write_kb":16891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":10508628,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"162'596","reported_seq":3023,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765441+0000","last_change":"2024-09-12T07:22:06.166247+0000","last_active":"2024-09-12T07:22:34.765441+0000","last_peered":"2024-09-12T07:22:34.765441+0000","last_clean":"2024-09-12T07:22:34.765441+0000","last_became_active":"2024-09-12T07:22:01.143359+0000","last_became_peered":"2024-09-12T07:22:01.143359+0000","last_unstale":"2024-09-12T07:22:34.765441+0000","last_undegraded":"2024-09-12T07:22:34.765441+0000","last_fullsized":"2024-09-12T07:22:34.765441+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:49.338904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001515553,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3463,"num_read_kb":13505,"num_write":637,"num_write_kb":16522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":8725508,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"152'338","reported_seq":1797,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765144+0000","last_change":"2024-09-12T07:22:31.247099+0000","last_active":"2024-09-12T07:22:34.765144+0000","last_peered":"2024-09-12T07:22:34.765144+0000","last_clean":"2024-09-12T07:22:34.765144+0000","last_became_active":"2024-09-12T07:22:31.245905+0000","last_became_peered":"2024-09-12T07:22:31.245905+0000","last_unstale":"2024-09-12T07:22:34.765144+0000","last_undegraded":"2024-09-12T07:22:34.765144+0000","last_fullsized":"2024-09-12T07:22:34.765144+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:10.350713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816669999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1472,"num_read_kb":27101,"num_write":327,"num_write_kb":18963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"163'454","reported_seq":7380,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764987+0000","last_change":"2024-09-12T07:22:06.164601+0000","last_active":"2024-09-12T07:22:34.764987+0000","last_peered":"2024-09-12T07:22:34.764987+0000","last_clean":"2024-09-12T07:22:34.764987+0000","last_became_active":"2024-09-12T07:22:01.143631+0000","last_became_peered":"2024-09-12T07:22:01.143631+0000","last_unstale":"2024-09-12T07:22:34.764987+0000","last_undegraded":"2024-09-12T07:22:34.764987+0000","last_fullsized":"2024-09-12T07:22:34.764987+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:34.629049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065451099999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7090,"num_read_kb":47369,"num_write":461,"num_write_kb":21538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":17766842,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"159'408","reported_seq":11210,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264074+0000","last_change":"2024-09-12T07:22:06.196502+0000","last_active":"2024-09-12T07:22:34.264074+0000","last_peered":"2024-09-12T07:22:34.264074+0000","last_clean":"2024-09-12T07:22:34.264074+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:22:34.264074+0000","last_undegraded":"2024-09-12T07:22:34.264074+0000","last_fullsized":"2024-09-12T07:22:34.264074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:48.940535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019415999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11863,"num_read_kb":47954,"num_write":410,"num_write_kb":15586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'476","reported_seq":19728,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765222+0000","last_change":"2024-09-12T07:22:06.165173+0000","last_active":"2024-09-12T07:22:34.765222+0000","last_peered":"2024-09-12T07:22:34.765222+0000","last_clean":"2024-09-12T07:22:34.765222+0000","last_became_active":"2024-09-12T07:22:01.143577+0000","last_became_peered":"2024-09-12T07:22:01.143577+0000","last_unstale":"2024-09-12T07:22:34.765222+0000","last_undegraded":"2024-09-12T07:22:34.765222+0000","last_fullsized":"2024-09-12T07:22:34.765222+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:12:53.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010515870000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":18947,"num_read_kb":82796,"num_write":553,"num_write_kb":44274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":53090062,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":323,"num_object_clones":0,"num_object_copies":646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":319,"num_read":260585,"num_read_kb":1190964,"num_write":20483,"num_write_kb":597347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5845,"num_bytes_recovered":432794818,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},"log_size":17959,"ondisk_log_size":17959,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56164,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315676,"statfs":{"total":96636764160,"available":96579252224,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33376,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338464,"statfs":{"total":96636764160,"available":96602587136,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55576,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316264,"statfs":{"total":96636764160,"available":96579854336,"internally_reserved":0,"allocated":2232320,"data_stored":1075567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51432,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320408,"statfs":{"total":96636764160,"available":96584097792,"internally_reserved":0,"allocated":2244608,"data_stored":1081409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69480,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302360,"statfs":{"total":96636764160,"available":96565616640,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":165,"seq":708669603848,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31008,"kb_used_data":2188,"kb_used_omap":23,"kb_used_meta":28776,"kb_avail":94340832,"statfs":{"total":96636764160,"available":96605011968,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24213,"internal_metadata":29466987},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55592,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316248,"statfs":{"total":96636764160,"available":96579837952,"internally_reserved":0,"allocated":2248704,"data_stored":1081428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64488,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307352,"statfs":{"total":96636764160,"available":96570728448,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:22:39.437 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T07:22:39.737 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:22:39.737 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":186,"fsid":"8412ed58-ac5e-4549-b523-1a58cf9f9429","created":"2024-09-12T07:19:10.328057+0000","modified":"2024-09-12T07:22:34.246048+0000","last_up_change":"2024-09-12T07:22:08.169187+0000","last_in_change":"2024-09-12T07:22:18.153362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T07:19:23.507151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T07:19:28.593632+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-12T07:19:40.555421+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":163,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"710c25c3-92d3-4b51-a7fc-670879faa69c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6817","nonce":4081287959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6819","nonce":4081287959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6823","nonce":4081287959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":4081287959},{"type":"v1","addr":"172.21.15.46:6821","nonce":4081287959}]},"public_addr":"172.21.15.46:6817/4081287959","cluster_addr":"172.21.15.46:6819/4081287959","heartbeat_back_addr":"172.21.15.46:6823/4081287959","heartbeat_front_addr":"172.21.15.46:6821/4081287959","state":["exists","up"]},{"osd":1,"uuid":"04479191-c52e-4b60-aa0d-3e911f869062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6825","nonce":3980488213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6827","nonce":3980488213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6831","nonce":3980488213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":3980488213},{"type":"v1","addr":"172.21.15.46:6829","nonce":3980488213}]},"public_addr":"172.21.15.46:6825/3980488213","cluster_addr":"172.21.15.46:6827/3980488213","heartbeat_back_addr":"172.21.15.46:6831/3980488213","heartbeat_front_addr":"172.21.15.46:6829/3980488213","state":["exists","up"]},{"osd":2,"uuid":"ba673e51-96aa-424b-a4df-26432d633ebe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":140,"up_from":165,"up_thru":172,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3239477423},{"type":"v1","addr":"172.21.15.46:6801","nonce":3239477423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3239477423},{"type":"v1","addr":"172.21.15.46:6803","nonce":3239477423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3239477423},{"type":"v1","addr":"172.21.15.46:6807","nonce":3239477423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3239477423},{"type":"v1","addr":"172.21.15.46:6805","nonce":3239477423}]},"public_addr":"172.21.15.46:6801/3239477423","cluster_addr":"172.21.15.46:6803/3239477423","heartbeat_back_addr":"172.21.15.46:6807/3239477423","heartbeat_front_addr":"172.21.15.46:6805/3239477423","state":["exists","up"]},{"osd":3,"uuid":"c516e59c-834c-4fb0-9ff8-bcc80b21d28a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6809","nonce":3848260327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6811","nonce":3848260327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6815","nonce":3848260327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3848260327},{"type":"v1","addr":"172.21.15.46:6813","nonce":3848260327}]},"public_addr":"172.21.15.46:6809/3848260327","cluster_addr":"172.21.15.46:6811/3848260327","heartbeat_back_addr":"172.21.15.46:6815/3848260327","heartbeat_front_addr":"172.21.15.46:6813/3848260327","state":["exists","up"]},{"osd":4,"uuid":"450a70cd-ebb2-48ed-a766-184f58e7032e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6817","nonce":3464256744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6819","nonce":3464256744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6823","nonce":3464256744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3464256744},{"type":"v1","addr":"172.21.15.64:6821","nonce":3464256744}]},"public_addr":"172.21.15.64:6817/3464256744","cluster_addr":"172.21.15.64:6819/3464256744","heartbeat_back_addr":"172.21.15.64:6823/3464256744","heartbeat_front_addr":"172.21.15.64:6821/3464256744","state":["exists","up"]},{"osd":5,"uuid":"a4efa333-bf57-45e7-a281-1d3bc9d2e82b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6825","nonce":1789414693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6827","nonce":1789414693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6831","nonce":1789414693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1789414693},{"type":"v1","addr":"172.21.15.64:6829","nonce":1789414693}]},"public_addr":"172.21.15.64:6825/1789414693","cluster_addr":"172.21.15.64:6827/1789414693","heartbeat_back_addr":"172.21.15.64:6831/1789414693","heartbeat_front_addr":"172.21.15.64:6829/1789414693","state":["exists","up"]},{"osd":6,"uuid":"bcaad352-f9ee-426c-bcbf-39c8d031aae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6801","nonce":2576774646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6804","nonce":2576774646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6813","nonce":2576774646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2576774646},{"type":"v1","addr":"172.21.15.64:6808","nonce":2576774646}]},"public_addr":"172.21.15.64:6801/2576774646","cluster_addr":"172.21.15.64:6804/2576774646","heartbeat_back_addr":"172.21.15.64:6813/2576774646","heartbeat_front_addr":"172.21.15.64:6808/2576774646","state":["exists","up"]},{"osd":7,"uuid":"97e5ef36-58a4-4667-9f9e-b8ced5904d69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6803","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6805","nonce":368918691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6807","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6809","nonce":368918691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6815","nonce":368918691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6811","nonce":368918691},{"type":"v1","addr":"172.21.15.64:6812","nonce":368918691}]},"public_addr":"172.21.15.64:6805/368918691","cluster_addr":"172.21.15.64:6809/368918691","heartbeat_back_addr":"172.21.15.64:6815/368918691","heartbeat_front_addr":"172.21.15.64:6812/368918691","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.862785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.059718+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.575165+0000","dead_epoch":141},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:18.847666+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.471904+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.061712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.541383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T07:19:19.072272+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.a","osds":[4,1]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T07:22:40.751 INFO:tasks.ceph:Scrubbing osd.0 2024-09-12T07:22:40.751 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-12T07:22:40.893 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:22:40.893 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T07:22:40.893 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:22:40.904 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-12T07:22:41.203 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 0 to deep-scrub 2024-09-12T07:22:41.216 INFO:tasks.ceph:Scrubbing osd.1 2024-09-12T07:22:41.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-12T07:22:41.343 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:22:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T07:22:41.343 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:22:41.354 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-12T07:22:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 1 to deep-scrub 2024-09-12T07:22:41.669 INFO:tasks.ceph:Scrubbing osd.2 2024-09-12T07:22:41.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-12T07:22:41.799 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:22:41.800 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-12T07:22:41.800 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:22:41.810 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-12T07:22:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 2 to deep-scrub 2024-09-12T07:22:42.126 INFO:tasks.ceph:Scrubbing osd.3 2024-09-12T07:22:42.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-12T07:22:42.253 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:22:42.253 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T07:22:42.253 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:22:42.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-12T07:22:42.562 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 3 to deep-scrub 2024-09-12T07:22:42.575 INFO:tasks.ceph:Scrubbing osd.4 2024-09-12T07:22:42.575 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-12T07:22:42.703 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:22:42.703 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T07:22:42.703 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:22:42.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-12T07:22:43.011 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 4 to deep-scrub 2024-09-12T07:22:43.024 INFO:tasks.ceph:Scrubbing osd.5 2024-09-12T07:22:43.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-12T07:22:43.156 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:22:43.156 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T07:22:43.156 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:22:43.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-12T07:22:43.465 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 5 to deep-scrub 2024-09-12T07:22:43.478 INFO:tasks.ceph:Scrubbing osd.6 2024-09-12T07:22:43.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-12T07:22:43.607 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:22:43.608 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T07:22:43.608 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:22:43.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-12T07:22:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 6 to deep-scrub 2024-09-12T07:22:43.932 INFO:tasks.ceph:Scrubbing osd.7 2024-09-12T07:22:43.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-12T07:22:44.061 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T07:22:44.062 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T07:22:44.062 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T07:22:44.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-12T07:22:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 7 to deep-scrub 2024-09-12T07:22:44.391 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:22:44.688 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:22:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:22:44.700 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-12T07:22:43.527099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":327,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":319,"num_read":260651,"num_read_kb":1191019,"num_write":20586,"num_write_kb":598977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5849,"num_bytes_recovered":432794875,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18051,"ondisk_log_size":18051,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417128,"kb_used_data":18680,"kb_used_omap":34,"kb_used_meta":398237,"kb_avail":754557592,"statfs":{"total":773094113280,"available":772666974208,"internally_reserved":0,"allocated":19128320,"data_stored":9831932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35370,"internal_metadata":407795158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002525"},"pg_stats":[{"pgid":"3.1f","version":"166'638","reported_seq":6785,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264251+0000","last_change":"2024-09-12T07:22:22.149919+0000","last_active":"2024-09-12T07:22:34.264251+0000","last_peered":"2024-09-12T07:22:34.264251+0000","last_clean":"2024-09-12T07:22:34.264251+0000","last_became_active":"2024-09-12T07:22:22.148401+0000","last_became_peered":"2024-09-12T07:22:22.148401+0000","last_unstale":"2024-09-12T07:22:34.264251+0000","last_undegraded":"2024-09-12T07:22:34.264251+0000","last_fullsized":"2024-09-12T07:22:34.264251+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:27:39.529993+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.029437784000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7771,"num_read_kb":29649,"num_write":783,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":2971697,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"161'489","reported_seq":2632,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707535+0000","last_change":"2024-09-12T07:22:06.175231+0000","last_active":"2024-09-12T07:22:34.707535+0000","last_peered":"2024-09-12T07:22:34.707535+0000","last_clean":"2024-09-12T07:22:34.707535+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:22:34.707535+0000","last_undegraded":"2024-09-12T07:22:34.707535+0000","last_fullsized":"2024-09-12T07:22:34.707535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:33:19.436850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026801759999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20338,"num_read_kb":38377,"num_write":424,"num_write_kb":22354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"163'468","reported_seq":21373,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765348+0000","last_change":"2024-09-12T07:22:22.151697+0000","last_active":"2024-09-12T07:22:34.765348+0000","last_peered":"2024-09-12T07:22:34.765348+0000","last_clean":"2024-09-12T07:22:34.765348+0000","last_became_active":"2024-09-12T07:22:22.149491+0000","last_became_peered":"2024-09-12T07:22:22.149491+0000","last_unstale":"2024-09-12T07:22:34.765348+0000","last_undegraded":"2024-09-12T07:22:34.765348+0000","last_fullsized":"2024-09-12T07:22:34.765348+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:51.871543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010058409999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":20924,"num_read_kb":39045,"num_write":507,"num_write_kb":15255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":23048347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"163'607","reported_seq":3893,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764874+0000","last_change":"2024-09-12T07:22:21.150401+0000","last_active":"2024-09-12T07:22:34.764874+0000","last_peered":"2024-09-12T07:22:34.764874+0000","last_clean":"2024-09-12T07:22:34.764874+0000","last_became_active":"2024-09-12T07:22:21.148942+0000","last_became_peered":"2024-09-12T07:22:21.148942+0000","last_unstale":"2024-09-12T07:22:34.764874+0000","last_undegraded":"2024-09-12T07:22:34.764874+0000","last_fullsized":"2024-09-12T07:22:34.764874+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:45.508340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014959909999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":3878,"num_read_kb":17362,"num_write":791,"num_write_kb":11625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":8199265,"num_keys_recovered":174,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"162'447","reported_seq":22008,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264233+0000","last_change":"2024-09-12T07:22:06.195928+0000","last_active":"2024-09-12T07:22:34.264233+0000","last_peered":"2024-09-12T07:22:34.264233+0000","last_clean":"2024-09-12T07:22:34.264233+0000","last_became_active":"2024-09-12T07:22:01.145404+0000","last_became_peered":"2024-09-12T07:22:01.145404+0000","last_unstale":"2024-09-12T07:22:34.264233+0000","last_undegraded":"2024-09-12T07:22:34.264233+0000","last_fullsized":"2024-09-12T07:22:34.264233+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:03.385436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014847230000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":21866,"num_read_kb":61051,"num_write":426,"num_write_kb":15720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":12377120,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'388","reported_seq":4868,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249862+0000","last_change":"2024-09-12T07:22:27.215825+0000","last_active":"2024-09-12T07:22:34.249862+0000","last_peered":"2024-09-12T07:22:34.249862+0000","last_clean":"2024-09-12T07:22:34.249862+0000","last_became_active":"2024-09-12T07:22:27.213519+0000","last_became_peered":"2024-09-12T07:22:27.213519+0000","last_unstale":"2024-09-12T07:22:34.249862+0000","last_undegraded":"2024-09-12T07:22:34.249862+0000","last_fullsized":"2024-09-12T07:22:34.249862+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:54.924110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022138280000000001,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4657,"num_read_kb":13211,"num_write":331,"num_write_kb":17325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":27687211,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"163'649","reported_seq":7971,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764092+0000","last_change":"2024-09-12T07:22:21.149665+0000","last_active":"2024-09-12T07:22:34.764092+0000","last_peered":"2024-09-12T07:22:34.764092+0000","last_clean":"2024-09-12T07:22:34.764092+0000","last_became_active":"2024-09-12T07:22:21.147583+0000","last_became_peered":"2024-09-12T07:22:21.147583+0000","last_unstale":"2024-09-12T07:22:34.764092+0000","last_undegraded":"2024-09-12T07:22:34.764092+0000","last_fullsized":"2024-09-12T07:22:34.764092+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:22.335622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023936500000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8436,"num_read_kb":50796,"num_write":779,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":7815100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"163'609","reported_seq":3818,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707368+0000","last_change":"2024-09-12T07:22:06.173059+0000","last_active":"2024-09-12T07:22:34.707368+0000","last_peered":"2024-09-12T07:22:34.707368+0000","last_clean":"2024-09-12T07:22:34.707368+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:22:34.707368+0000","last_undegraded":"2024-09-12T07:22:34.707368+0000","last_fullsized":"2024-09-12T07:22:34.707368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:17.727732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099897500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6008,"num_read_kb":52191,"num_write":744,"num_write_kb":22227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"162'653","reported_seq":7128,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264260+0000","last_change":"2024-09-12T07:22:22.149915+0000","last_active":"2024-09-12T07:22:34.264260+0000","last_peered":"2024-09-12T07:22:34.264260+0000","last_clean":"2024-09-12T07:22:34.264260+0000","last_became_active":"2024-09-12T07:22:22.148392+0000","last_became_peered":"2024-09-12T07:22:22.148392+0000","last_unstale":"2024-09-12T07:22:34.264260+0000","last_undegraded":"2024-09-12T07:22:34.264260+0000","last_fullsized":"2024-09-12T07:22:34.264260+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:29.255739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002496505,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8906,"num_read_kb":19832,"num_write":605,"num_write_kb":22990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":20211745,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"163'375","reported_seq":1927,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765128+0000","last_change":"2024-09-12T07:22:21.149525+0000","last_active":"2024-09-12T07:22:34.765128+0000","last_peered":"2024-09-12T07:22:34.765128+0000","last_clean":"2024-09-12T07:22:34.765128+0000","last_became_active":"2024-09-12T07:22:21.148026+0000","last_became_peered":"2024-09-12T07:22:21.148026+0000","last_unstale":"2024-09-12T07:22:34.765128+0000","last_undegraded":"2024-09-12T07:22:34.765128+0000","last_fullsized":"2024-09-12T07:22:34.765128+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:15.887884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017138209999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1348,"num_read_kb":18539,"num_write":421,"num_write_kb":14413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":10121363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"152'344","reported_seq":7171,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765014+0000","last_change":"2024-09-12T07:22:06.164819+0000","last_active":"2024-09-12T07:22:34.765014+0000","last_peered":"2024-09-12T07:22:34.765014+0000","last_clean":"2024-09-12T07:22:34.765014+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:22:34.765014+0000","last_undegraded":"2024-09-12T07:22:34.765014+0000","last_fullsized":"2024-09-12T07:22:34.765014+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:45.370726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080767000000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7070,"num_read_kb":53310,"num_write":299,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"160'391","reported_seq":5850,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.250009+0000","last_change":"2024-09-12T07:22:27.214683+0000","last_active":"2024-09-12T07:22:34.250009+0000","last_peered":"2024-09-12T07:22:34.250009+0000","last_clean":"2024-09-12T07:22:34.250009+0000","last_became_active":"2024-09-12T07:22:27.212420+0000","last_became_peered":"2024-09-12T07:22:27.212420+0000","last_unstale":"2024-09-12T07:22:34.250009+0000","last_undegraded":"2024-09-12T07:22:34.250009+0000","last_fullsized":"2024-09-12T07:22:34.250009+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:16.234305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097451600000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5380,"num_read_kb":60804,"num_write":390,"num_write_kb":30252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":31149476,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764999+0000","last_change":"2024-09-12T07:22:31.308130+0000","last_active":"2024-09-12T07:22:34.764999+0000","last_peered":"2024-09-12T07:22:34.764999+0000","last_clean":"2024-09-12T07:22:34.764999+0000","last_became_active":"2024-09-12T07:22:31.307821+0000","last_became_peered":"2024-09-12T07:22:31.307821+0000","last_unstale":"2024-09-12T07:22:34.764999+0000","last_undegraded":"2024-09-12T07:22:34.764999+0000","last_fullsized":"2024-09-12T07:22:34.764999+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:45.276396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"186'656","reported_seq":6291,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:41.783694+0000","last_change":"2024-09-12T07:22:41.783694+0000","last_active":"2024-09-12T07:22:41.783694+0000","last_peered":"2024-09-12T07:22:41.783694+0000","last_clean":"2024-09-12T07:22:41.783694+0000","last_became_active":"2024-09-12T07:22:21.152036+0000","last_became_peered":"2024-09-12T07:22:21.152036+0000","last_unstale":"2024-09-12T07:22:41.783694+0000","last_undegraded":"2024-09-12T07:22:41.783694+0000","last_fullsized":"2024-09-12T07:22:41.783694+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'656","last_scrub_stamp":"2024-09-12T07:22:41.783599+0000","last_deep_scrub":"186'656","last_deep_scrub_stamp":"2024-09-12T07:22:41.783599+0000","last_clean_scrub_stamp":"2024-09-12T07:22:41.783599+0000","objects_scrubbed":17,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:37:07.820257+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0014037019999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6009,"num_read_kb":23012,"num_write":704,"num_write_kb":11415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":1480654,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764229+0000","last_change":"2024-09-12T07:22:18.173512+0000","last_active":"2024-09-12T07:22:34.764229+0000","last_peered":"2024-09-12T07:22:34.764229+0000","last_clean":"2024-09-12T07:22:34.764229+0000","last_became_active":"2024-09-12T07:22:18.173148+0000","last_became_peered":"2024-09-12T07:22:18.173148+0000","last_unstale":"2024-09-12T07:22:34.764229+0000","last_undegraded":"2024-09-12T07:22:34.764229+0000","last_fullsized":"2024-09-12T07:22:34.764229+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:07.135570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"162'679","reported_seq":2209,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707544+0000","last_change":"2024-09-12T07:22:06.174857+0000","last_active":"2024-09-12T07:22:34.707544+0000","last_peered":"2024-09-12T07:22:34.707544+0000","last_clean":"2024-09-12T07:22:34.707544+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:22:34.707544+0000","last_undegraded":"2024-09-12T07:22:34.707544+0000","last_fullsized":"2024-09-12T07:22:34.707544+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:16.561231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025712959999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3057,"num_read_kb":14289,"num_write":898,"num_write_kb":11329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":348,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764698+0000","last_change":"2024-09-12T07:19:33.033345+0000","last_active":"2024-09-12T07:22:34.764698+0000","last_peered":"2024-09-12T07:22:34.764698+0000","last_clean":"2024-09-12T07:22:34.764698+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:22:34.764698+0000","last_undegraded":"2024-09-12T07:22:34.764698+0000","last_fullsized":"2024-09-12T07:22:34.764698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:41.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"166'506","reported_seq":3850,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764186+0000","last_change":"2024-09-12T07:22:21.149345+0000","last_active":"2024-09-12T07:22:34.764186+0000","last_peered":"2024-09-12T07:22:34.764186+0000","last_clean":"2024-09-12T07:22:34.764186+0000","last_became_active":"2024-09-12T07:22:21.147703+0000","last_became_peered":"2024-09-12T07:22:21.147703+0000","last_unstale":"2024-09-12T07:22:34.764186+0000","last_undegraded":"2024-09-12T07:22:34.764186+0000","last_fullsized":"2024-09-12T07:22:34.764186+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:05.437283+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.023667932999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4682,"num_read_kb":14844,"num_write":579,"num_write_kb":8813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6163290,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:42.095173+0000","last_change":"2024-09-12T07:22:42.095173+0000","last_active":"2024-09-12T07:22:42.095173+0000","last_peered":"2024-09-12T07:22:42.095173+0000","last_clean":"2024-09-12T07:22:42.095173+0000","last_became_active":"2024-09-12T07:22:18.169285+0000","last_became_peered":"2024-09-12T07:22:18.169285+0000","last_unstale":"2024-09-12T07:22:42.095173+0000","last_undegraded":"2024-09-12T07:22:42.095173+0000","last_fullsized":"2024-09-12T07:22:42.095173+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:22:42.095102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:22:42.095102+0000","last_clean_scrub_stamp":"2024-09-12T07:22:42.095102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:53:43.674678+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"166'565","reported_seq":10840,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765459+0000","last_change":"2024-09-12T07:22:22.153784+0000","last_active":"2024-09-12T07:22:34.765459+0000","last_peered":"2024-09-12T07:22:34.765459+0000","last_clean":"2024-09-12T07:22:34.765459+0000","last_became_active":"2024-09-12T07:22:22.151818+0000","last_became_peered":"2024-09-12T07:22:22.151818+0000","last_unstale":"2024-09-12T07:22:34.765459+0000","last_undegraded":"2024-09-12T07:22:34.765459+0000","last_fullsized":"2024-09-12T07:22:34.765459+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:02.130727+0000","scrub_duration":0,"objects_trimmed":18,"snaptrim_duration":0.023276249999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11624,"num_read_kb":77562,"num_write":595,"num_write_kb":25411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":22089334,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"163'503","reported_seq":4495,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249801+0000","last_change":"2024-09-12T07:22:27.215301+0000","last_active":"2024-09-12T07:22:34.249801+0000","last_peered":"2024-09-12T07:22:34.249801+0000","last_clean":"2024-09-12T07:22:34.249801+0000","last_became_active":"2024-09-12T07:22:27.213073+0000","last_became_peered":"2024-09-12T07:22:27.213073+0000","last_unstale":"2024-09-12T07:22:34.249801+0000","last_undegraded":"2024-09-12T07:22:34.249801+0000","last_fullsized":"2024-09-12T07:22:34.249801+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:10.316262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018780750000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":3159,"num_read_kb":12201,"num_write":460,"num_write_kb":10579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":2603172,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":48,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765232+0000","last_change":"2024-09-12T07:22:19.200232+0000","last_active":"2024-09-12T07:22:34.765232+0000","last_peered":"2024-09-12T07:22:34.765232+0000","last_clean":"2024-09-12T07:22:34.765232+0000","last_became_active":"2024-09-12T07:22:19.178595+0000","last_became_peered":"2024-09-12T07:22:19.178595+0000","last_unstale":"2024-09-12T07:22:34.765232+0000","last_undegraded":"2024-09-12T07:22:34.765232+0000","last_fullsized":"2024-09-12T07:22:34.765232+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:58.776762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"166'781","reported_seq":3325,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764213+0000","last_change":"2024-09-12T07:22:21.150766+0000","last_active":"2024-09-12T07:22:34.764213+0000","last_peered":"2024-09-12T07:22:34.764213+0000","last_clean":"2024-09-12T07:22:34.764213+0000","last_became_active":"2024-09-12T07:22:21.147758+0000","last_became_peered":"2024-09-12T07:22:21.147758+0000","last_unstale":"2024-09-12T07:22:34.764213+0000","last_undegraded":"2024-09-12T07:22:34.764213+0000","last_fullsized":"2024-09-12T07:22:34.764213+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:15.197467+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.033858052999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5450,"num_read_kb":11647,"num_write":1052,"num_write_kb":13386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":9483314,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":299,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249699+0000","last_change":"2024-09-12T07:22:09.183569+0000","last_active":"2024-09-12T07:22:34.249699+0000","last_peered":"2024-09-12T07:22:34.249699+0000","last_clean":"2024-09-12T07:22:34.249699+0000","last_became_active":"2024-09-12T07:22:09.182547+0000","last_became_peered":"2024-09-12T07:22:09.182547+0000","last_unstale":"2024-09-12T07:22:34.249699+0000","last_undegraded":"2024-09-12T07:22:34.249699+0000","last_fullsized":"2024-09-12T07:22:34.249699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:52.520157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"162'406","reported_seq":10545,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264069+0000","last_change":"2024-09-12T07:22:06.195337+0000","last_active":"2024-09-12T07:22:34.264069+0000","last_peered":"2024-09-12T07:22:34.264069+0000","last_clean":"2024-09-12T07:22:34.264069+0000","last_became_active":"2024-09-12T07:22:01.146421+0000","last_became_peered":"2024-09-12T07:22:01.146421+0000","last_unstale":"2024-09-12T07:22:34.264069+0000","last_undegraded":"2024-09-12T07:22:34.264069+0000","last_fullsized":"2024-09-12T07:22:34.264069+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:38.812988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094442799999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11174,"num_read_kb":59795,"num_write":376,"num_write_kb":27716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":22543830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":360,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764760+0000","last_change":"2024-09-12T07:22:18.171696+0000","last_active":"2024-09-12T07:22:34.764760+0000","last_peered":"2024-09-12T07:22:34.764760+0000","last_clean":"2024-09-12T07:22:34.764760+0000","last_became_active":"2024-09-12T07:22:18.171266+0000","last_became_peered":"2024-09-12T07:22:18.171266+0000","last_unstale":"2024-09-12T07:22:34.764760+0000","last_undegraded":"2024-09-12T07:22:34.764760+0000","last_fullsized":"2024-09-12T07:22:34.764760+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:26.361507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"162'1062","reported_seq":6945,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707508+0000","last_change":"2024-09-12T07:22:22.152024+0000","last_active":"2024-09-12T07:22:34.707508+0000","last_peered":"2024-09-12T07:22:34.707508+0000","last_clean":"2024-09-12T07:22:34.707508+0000","last_became_active":"2024-09-12T07:22:22.150316+0000","last_became_peered":"2024-09-12T07:22:22.150316+0000","last_unstale":"2024-09-12T07:22:34.707508+0000","last_undegraded":"2024-09-12T07:22:34.707508+0000","last_fullsized":"2024-09-12T07:22:34.707508+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:57:13.384646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021189680000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8933,"num_read_kb":59814,"num_write":1570,"num_write_kb":27462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":14379761,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":45,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764860+0000","last_change":"2024-09-12T07:22:19.178038+0000","last_active":"2024-09-12T07:22:34.764860+0000","last_peered":"2024-09-12T07:22:34.764860+0000","last_clean":"2024-09-12T07:22:34.764860+0000","last_became_active":"2024-09-12T07:22:19.170715+0000","last_became_peered":"2024-09-12T07:22:19.170715+0000","last_unstale":"2024-09-12T07:22:34.764860+0000","last_undegraded":"2024-09-12T07:22:34.764860+0000","last_fullsized":"2024-09-12T07:22:34.764860+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:29.198356+0000","last_clean_scrub_stamp":"2024-09-12T07:19:29.198356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:28.982893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":439,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764900+0000","last_change":"2024-09-12T07:19:25.193493+0000","last_active":"2024-09-12T07:22:34.764900+0000","last_peered":"2024-09-12T07:22:34.764900+0000","last_clean":"2024-09-12T07:22:34.764900+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:22:34.764900+0000","last_undegraded":"2024-09-12T07:22:34.764900+0000","last_fullsized":"2024-09-12T07:22:34.764900+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:24.181227+0000","last_clean_scrub_stamp":"2024-09-12T07:19:24.181227+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:20:03.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"162'962","reported_seq":3107,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707372+0000","last_change":"2024-09-12T07:22:21.151517+0000","last_active":"2024-09-12T07:22:34.707372+0000","last_peered":"2024-09-12T07:22:34.707372+0000","last_clean":"2024-09-12T07:22:34.707372+0000","last_became_active":"2024-09-12T07:22:21.150083+0000","last_became_peered":"2024-09-12T07:22:21.150083+0000","last_unstale":"2024-09-12T07:22:34.707372+0000","last_undegraded":"2024-09-12T07:22:34.707372+0000","last_fullsized":"2024-09-12T07:22:34.707372+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":962,"log_dups_size":0,"ondisk_log_size":962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:38.471826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011038020000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5044,"num_read_kb":23221,"num_write":1448,"num_write_kb":24489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":21462395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"163'549","reported_seq":5426,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249824+0000","last_change":"2024-09-12T07:22:27.215048+0000","last_active":"2024-09-12T07:22:34.249824+0000","last_peered":"2024-09-12T07:22:34.249824+0000","last_clean":"2024-09-12T07:22:34.249824+0000","last_became_active":"2024-09-12T07:22:27.212942+0000","last_became_peered":"2024-09-12T07:22:27.212942+0000","last_unstale":"2024-09-12T07:22:34.249824+0000","last_undegraded":"2024-09-12T07:22:34.249824+0000","last_fullsized":"2024-09-12T07:22:34.249824+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:59.386875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021244950000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7563,"num_read_kb":32357,"num_write":525,"num_write_kb":20455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1710555,"num_keys_recovered":41,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"162'540","reported_seq":6677,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.249766+0000","last_change":"2024-09-12T07:22:27.215838+0000","last_active":"2024-09-12T07:22:34.249766+0000","last_peered":"2024-09-12T07:22:34.249766+0000","last_clean":"2024-09-12T07:22:34.249766+0000","last_became_active":"2024-09-12T07:22:27.213533+0000","last_became_peered":"2024-09-12T07:22:27.213533+0000","last_unstale":"2024-09-12T07:22:34.249766+0000","last_undegraded":"2024-09-12T07:22:34.249766+0000","last_fullsized":"2024-09-12T07:22:34.249766+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:38.800561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069883599999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":6247,"num_read_kb":49000,"num_write":522,"num_write_kb":25535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":42565291,"num_keys_recovered":104,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"162'692","reported_seq":3567,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.707411+0000","last_change":"2024-09-12T07:22:06.173724+0000","last_active":"2024-09-12T07:22:34.707411+0000","last_peered":"2024-09-12T07:22:34.707411+0000","last_clean":"2024-09-12T07:22:34.707411+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:22:34.707411+0000","last_undegraded":"2024-09-12T07:22:34.707411+0000","last_fullsized":"2024-09-12T07:22:34.707411+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:30.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015261610000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6009,"num_read_kb":21069,"num_write":727,"num_write_kb":17071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"162'445","reported_seq":9922,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764933+0000","last_change":"2024-09-12T07:22:22.151400+0000","last_active":"2024-09-12T07:22:34.764933+0000","last_peered":"2024-09-12T07:22:34.764933+0000","last_clean":"2024-09-12T07:22:34.764933+0000","last_became_active":"2024-09-12T07:22:22.149715+0000","last_became_peered":"2024-09-12T07:22:22.149715+0000","last_unstale":"2024-09-12T07:22:34.764933+0000","last_undegraded":"2024-09-12T07:22:34.764933+0000","last_fullsized":"2024-09-12T07:22:34.764933+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:51.662782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016506240000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11019,"num_read_kb":49685,"num_write":382,"num_write_kb":15924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":18973449,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"161'769","reported_seq":3640,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765503+0000","last_change":"2024-09-12T07:22:22.153946+0000","last_active":"2024-09-12T07:22:34.765503+0000","last_peered":"2024-09-12T07:22:34.765503+0000","last_clean":"2024-09-12T07:22:34.765503+0000","last_became_active":"2024-09-12T07:22:22.152455+0000","last_became_peered":"2024-09-12T07:22:22.152455+0000","last_unstale":"2024-09-12T07:22:34.765503+0000","last_undegraded":"2024-09-12T07:22:34.765503+0000","last_fullsized":"2024-09-12T07:22:34.765503+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:17.209337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014515540000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5118,"num_read_kb":30709,"num_write":1176,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":7370075,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"162'531","reported_seq":5409,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765193+0000","last_change":"2024-09-12T07:22:06.166758+0000","last_active":"2024-09-12T07:22:34.765193+0000","last_peered":"2024-09-12T07:22:34.765193+0000","last_clean":"2024-09-12T07:22:34.765193+0000","last_became_active":"2024-09-12T07:22:00.145108+0000","last_became_peered":"2024-09-12T07:22:00.145108+0000","last_unstale":"2024-09-12T07:22:34.765193+0000","last_undegraded":"2024-09-12T07:22:34.765193+0000","last_fullsized":"2024-09-12T07:22:34.765193+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:31.569737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801299999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6080,"num_read_kb":38867,"num_write":581,"num_write_kb":16891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":10508628,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"162'596","reported_seq":3023,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765441+0000","last_change":"2024-09-12T07:22:06.166247+0000","last_active":"2024-09-12T07:22:34.765441+0000","last_peered":"2024-09-12T07:22:34.765441+0000","last_clean":"2024-09-12T07:22:34.765441+0000","last_became_active":"2024-09-12T07:22:01.143359+0000","last_became_peered":"2024-09-12T07:22:01.143359+0000","last_unstale":"2024-09-12T07:22:34.765441+0000","last_undegraded":"2024-09-12T07:22:34.765441+0000","last_fullsized":"2024-09-12T07:22:34.765441+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:49.338904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001515553,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3463,"num_read_kb":13505,"num_write":637,"num_write_kb":16522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":8725508,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"152'338","reported_seq":1797,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765144+0000","last_change":"2024-09-12T07:22:31.247099+0000","last_active":"2024-09-12T07:22:34.765144+0000","last_peered":"2024-09-12T07:22:34.765144+0000","last_clean":"2024-09-12T07:22:34.765144+0000","last_became_active":"2024-09-12T07:22:31.245905+0000","last_became_peered":"2024-09-12T07:22:31.245905+0000","last_unstale":"2024-09-12T07:22:34.765144+0000","last_undegraded":"2024-09-12T07:22:34.765144+0000","last_fullsized":"2024-09-12T07:22:34.765144+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:10.350713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816669999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1472,"num_read_kb":27101,"num_write":327,"num_write_kb":18963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"163'454","reported_seq":7380,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.764987+0000","last_change":"2024-09-12T07:22:06.164601+0000","last_active":"2024-09-12T07:22:34.764987+0000","last_peered":"2024-09-12T07:22:34.764987+0000","last_clean":"2024-09-12T07:22:34.764987+0000","last_became_active":"2024-09-12T07:22:01.143631+0000","last_became_peered":"2024-09-12T07:22:01.143631+0000","last_unstale":"2024-09-12T07:22:34.764987+0000","last_undegraded":"2024-09-12T07:22:34.764987+0000","last_fullsized":"2024-09-12T07:22:34.764987+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:34.629049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065451099999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7090,"num_read_kb":47369,"num_write":461,"num_write_kb":21538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":17766842,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"159'408","reported_seq":11210,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.264074+0000","last_change":"2024-09-12T07:22:06.196502+0000","last_active":"2024-09-12T07:22:34.264074+0000","last_peered":"2024-09-12T07:22:34.264074+0000","last_clean":"2024-09-12T07:22:34.264074+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:22:34.264074+0000","last_undegraded":"2024-09-12T07:22:34.264074+0000","last_fullsized":"2024-09-12T07:22:34.264074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:48.940535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019415999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11863,"num_read_kb":47954,"num_write":410,"num_write_kb":15586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'476","reported_seq":19728,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:34.765222+0000","last_change":"2024-09-12T07:22:06.165173+0000","last_active":"2024-09-12T07:22:34.765222+0000","last_peered":"2024-09-12T07:22:34.765222+0000","last_clean":"2024-09-12T07:22:34.765222+0000","last_became_active":"2024-09-12T07:22:01.143577+0000","last_became_peered":"2024-09-12T07:22:01.143577+0000","last_unstale":"2024-09-12T07:22:34.765222+0000","last_undegraded":"2024-09-12T07:22:34.765222+0000","last_fullsized":"2024-09-12T07:22:34.765222+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:19:41.430913+0000","last_clean_scrub_stamp":"2024-09-12T07:19:41.430913+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:12:53.621045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010515870000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":18947,"num_read_kb":82796,"num_write":553,"num_write_kb":44274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":53090062,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":323,"num_object_clones":0,"num_object_copies":646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":319,"num_read":260585,"num_read_kb":1190964,"num_write":20483,"num_write_kb":597347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5845,"num_bytes_recovered":432794818,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},"log_size":17976,"ondisk_log_size":17976,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56164,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315676,"statfs":{"total":96636764160,"available":96579252224,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33376,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338464,"statfs":{"total":96636764160,"available":96602587136,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55592,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316248,"statfs":{"total":96636764160,"available":96579837952,"internally_reserved":0,"allocated":2244608,"data_stored":1081409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51432,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320408,"statfs":{"total":96636764160,"available":96584097792,"internally_reserved":0,"allocated":2244608,"data_stored":1081409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69480,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302360,"statfs":{"total":96636764160,"available":96565616640,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":165,"seq":708669603849,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31008,"kb_used_data":2188,"kb_used_omap":23,"kb_used_meta":28776,"kb_avail":94340832,"statfs":{"total":96636764160,"available":96605011968,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24213,"internal_metadata":29466987},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55588,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316252,"statfs":{"total":96636764160,"available":96579842048,"internally_reserved":0,"allocated":2248704,"data_stored":1081428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64488,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307352,"statfs":{"total":96636764160,"available":96570728448,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:22:44.705 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.705 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.706 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.706 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.706 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.706 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.706 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.706 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.706 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.706 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.707 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.707 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.707 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-12T07:19:29.198356+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.707 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-12T07:19:29.198356+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.707 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.707 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-12T07:19:29.198356+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.707 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.707 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.708 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.708 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-12T07:19:29.198356+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.708 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.708 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-12T07:19:29.198356+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.708 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.708 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-12T07:19:29.198356+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.708 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.708 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-12T07:19:29.198356+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.708 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-12T07:19:24.181227+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=24, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.708 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.709 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.709 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.709 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.709 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.709 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.709 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.709 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.709 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.710 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.710 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.710 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-12T07:19:41.430913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=19, tm_sec=41, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=7, tm_min=22, tm_sec=39, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T07:22:44.710 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-12T07:23:04.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T07:23:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T07:23:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T07:23:05.039 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-12T07:23:03.532003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":327,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":319,"num_read":260651,"num_read_kb":1191019,"num_write":20586,"num_write_kb":598977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5849,"num_bytes_recovered":432794875,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18361,"ondisk_log_size":18361,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":404600,"kb_used_data":18680,"kb_used_omap":34,"kb_used_meta":385757,"kb_avail":754570120,"statfs":{"total":773094113280,"available":772679802880,"internally_reserved":0,"allocated":19128320,"data_stored":9831932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35370,"internal_metadata":395015638},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002820"},"pg_stats":[{"pgid":"3.1f","version":"186'650","reported_seq":6805,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:46.891235+0000","last_change":"2024-09-12T07:22:46.891235+0000","last_active":"2024-09-12T07:22:46.891235+0000","last_peered":"2024-09-12T07:22:46.891235+0000","last_clean":"2024-09-12T07:22:46.891235+0000","last_became_active":"2024-09-12T07:22:22.148401+0000","last_became_peered":"2024-09-12T07:22:22.148401+0000","last_unstale":"2024-09-12T07:22:46.891235+0000","last_undegraded":"2024-09-12T07:22:46.891235+0000","last_fullsized":"2024-09-12T07:22:46.891235+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'650","last_scrub_stamp":"2024-09-12T07:22:46.891143+0000","last_deep_scrub":"186'650","last_deep_scrub_stamp":"2024-09-12T07:22:46.891143+0000","last_clean_scrub_stamp":"2024-09-12T07:22:46.891143+0000","objects_scrubbed":12,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:21:24.578773+0000","scrub_duration":9,"objects_trimmed":10,"snaptrim_duration":0.029437784000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7771,"num_read_kb":29649,"num_write":783,"num_write_kb":8051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":2971697,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"186'498","reported_seq":2649,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:46.099958+0000","last_change":"2024-09-12T07:22:46.099958+0000","last_active":"2024-09-12T07:22:46.099958+0000","last_peered":"2024-09-12T07:22:46.099958+0000","last_clean":"2024-09-12T07:22:46.099958+0000","last_became_active":"2024-09-12T07:20:07.180716+0000","last_became_peered":"2024-09-12T07:20:07.180716+0000","last_unstale":"2024-09-12T07:22:46.099958+0000","last_undegraded":"2024-09-12T07:22:46.099958+0000","last_fullsized":"2024-09-12T07:22:46.099958+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'498","last_scrub_stamp":"2024-09-12T07:22:46.099860+0000","last_deep_scrub":"186'498","last_deep_scrub_stamp":"2024-09-12T07:22:46.099860+0000","last_clean_scrub_stamp":"2024-09-12T07:22:46.099860+0000","objects_scrubbed":9,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:00.173062+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0026801759999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20338,"num_read_kb":38377,"num_write":424,"num_write_kb":22354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":16853430,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"186'480","reported_seq":21393,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:49.360835+0000","last_change":"2024-09-12T07:22:49.360835+0000","last_active":"2024-09-12T07:22:49.360835+0000","last_peered":"2024-09-12T07:22:49.360835+0000","last_clean":"2024-09-12T07:22:49.360835+0000","last_became_active":"2024-09-12T07:22:22.149491+0000","last_became_peered":"2024-09-12T07:22:22.149491+0000","last_unstale":"2024-09-12T07:22:49.360835+0000","last_undegraded":"2024-09-12T07:22:49.360835+0000","last_fullsized":"2024-09-12T07:22:49.360835+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'480","last_scrub_stamp":"2024-09-12T07:22:49.360735+0000","last_deep_scrub":"186'480","last_deep_scrub_stamp":"2024-09-12T07:22:49.360735+0000","last_clean_scrub_stamp":"2024-09-12T07:22:49.360735+0000","objects_scrubbed":12,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:56.600109+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0010058409999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":20924,"num_read_kb":39045,"num_write":507,"num_write_kb":15255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":23048347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"186'616","reported_seq":3910,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:50.326418+0000","last_change":"2024-09-12T07:22:50.326418+0000","last_active":"2024-09-12T07:22:50.326418+0000","last_peered":"2024-09-12T07:22:50.326418+0000","last_clean":"2024-09-12T07:22:50.326418+0000","last_became_active":"2024-09-12T07:22:21.148942+0000","last_became_peered":"2024-09-12T07:22:21.148942+0000","last_unstale":"2024-09-12T07:22:50.326418+0000","last_undegraded":"2024-09-12T07:22:50.326418+0000","last_fullsized":"2024-09-12T07:22:50.326418+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'616","last_scrub_stamp":"2024-09-12T07:22:50.326294+0000","last_deep_scrub":"186'616","last_deep_scrub_stamp":"2024-09-12T07:22:50.326294+0000","last_clean_scrub_stamp":"2024-09-12T07:22:50.326294+0000","objects_scrubbed":9,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:38.937147+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0014959909999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":3878,"num_read_kb":17362,"num_write":791,"num_write_kb":11625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":8199265,"num_keys_recovered":174,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"186'453","reported_seq":22022,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:45.890357+0000","last_change":"2024-09-12T07:22:45.890357+0000","last_active":"2024-09-12T07:22:45.890357+0000","last_peered":"2024-09-12T07:22:45.890357+0000","last_clean":"2024-09-12T07:22:45.890357+0000","last_became_active":"2024-09-12T07:22:01.145404+0000","last_became_peered":"2024-09-12T07:22:01.145404+0000","last_unstale":"2024-09-12T07:22:45.890357+0000","last_undegraded":"2024-09-12T07:22:45.890357+0000","last_fullsized":"2024-09-12T07:22:45.890357+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'453","last_scrub_stamp":"2024-09-12T07:22:45.890253+0000","last_deep_scrub":"186'453","last_deep_scrub_stamp":"2024-09-12T07:22:45.890253+0000","last_clean_scrub_stamp":"2024-09-12T07:22:45.890253+0000","objects_scrubbed":6,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.387241+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0014847230000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":21866,"num_read_kb":61051,"num_write":426,"num_write_kb":15720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":12377120,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"186'402","reported_seq":4890,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:47.378158+0000","last_change":"2024-09-12T07:22:47.378158+0000","last_active":"2024-09-12T07:22:47.378158+0000","last_peered":"2024-09-12T07:22:47.378158+0000","last_clean":"2024-09-12T07:22:47.378158+0000","last_became_active":"2024-09-12T07:22:27.213519+0000","last_became_peered":"2024-09-12T07:22:27.213519+0000","last_unstale":"2024-09-12T07:22:47.378158+0000","last_undegraded":"2024-09-12T07:22:47.378158+0000","last_fullsized":"2024-09-12T07:22:47.378158+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'402","last_scrub_stamp":"2024-09-12T07:22:47.378046+0000","last_deep_scrub":"186'402","last_deep_scrub_stamp":"2024-09-12T07:22:47.378046+0000","last_clean_scrub_stamp":"2024-09-12T07:22:47.378046+0000","objects_scrubbed":14,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:56.136791+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0022138280000000001,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4657,"num_read_kb":13211,"num_write":331,"num_write_kb":17325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":27687211,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"186'655","reported_seq":7985,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:46.129027+0000","last_change":"2024-09-12T07:22:46.129027+0000","last_active":"2024-09-12T07:22:46.129027+0000","last_peered":"2024-09-12T07:22:46.129027+0000","last_clean":"2024-09-12T07:22:46.129027+0000","last_became_active":"2024-09-12T07:22:21.147583+0000","last_became_peered":"2024-09-12T07:22:21.147583+0000","last_unstale":"2024-09-12T07:22:46.129027+0000","last_undegraded":"2024-09-12T07:22:46.129027+0000","last_fullsized":"2024-09-12T07:22:46.129027+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'655","last_scrub_stamp":"2024-09-12T07:22:46.128917+0000","last_deep_scrub":"186'655","last_deep_scrub_stamp":"2024-09-12T07:22:46.128917+0000","last_clean_scrub_stamp":"2024-09-12T07:22:46.128917+0000","objects_scrubbed":6,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:58:43.024595+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0023936500000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8436,"num_read_kb":50796,"num_write":779,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":7815100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"186'615","reported_seq":3832,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:48.109145+0000","last_change":"2024-09-12T07:22:48.109145+0000","last_active":"2024-09-12T07:22:48.109145+0000","last_peered":"2024-09-12T07:22:48.109145+0000","last_clean":"2024-09-12T07:22:48.109145+0000","last_became_active":"2024-09-12T07:20:07.181292+0000","last_became_peered":"2024-09-12T07:20:07.181292+0000","last_unstale":"2024-09-12T07:22:48.109145+0000","last_undegraded":"2024-09-12T07:22:48.109145+0000","last_fullsized":"2024-09-12T07:22:48.109145+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'615","last_scrub_stamp":"2024-09-12T07:22:48.109022+0000","last_deep_scrub":"186'615","last_deep_scrub_stamp":"2024-09-12T07:22:48.109022+0000","last_clean_scrub_stamp":"2024-09-12T07:22:48.109022+0000","objects_scrubbed":6,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:00:31.476254+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00099897500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6008,"num_read_kb":52191,"num_write":744,"num_write_kb":22227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":237454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"186'660","reported_seq":7143,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:42.794591+0000","last_change":"2024-09-12T07:22:42.794591+0000","last_active":"2024-09-12T07:22:42.794591+0000","last_peered":"2024-09-12T07:22:42.794591+0000","last_clean":"2024-09-12T07:22:42.794591+0000","last_became_active":"2024-09-12T07:22:22.148392+0000","last_became_peered":"2024-09-12T07:22:22.148392+0000","last_unstale":"2024-09-12T07:22:42.794591+0000","last_undegraded":"2024-09-12T07:22:42.794591+0000","last_fullsized":"2024-09-12T07:22:42.794591+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'660","last_scrub_stamp":"2024-09-12T07:22:42.794495+0000","last_deep_scrub":"186'660","last_deep_scrub_stamp":"2024-09-12T07:22:42.794495+0000","last_clean_scrub_stamp":"2024-09-12T07:22:42.794495+0000","objects_scrubbed":7,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:59:24.257417+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.002496505,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8906,"num_read_kb":19832,"num_write":605,"num_write_kb":22990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":20211745,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"186'388","reported_seq":1948,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:48.383760+0000","last_change":"2024-09-12T07:22:48.383760+0000","last_active":"2024-09-12T07:22:48.383760+0000","last_peered":"2024-09-12T07:22:48.383760+0000","last_clean":"2024-09-12T07:22:48.383760+0000","last_became_active":"2024-09-12T07:22:21.148026+0000","last_became_peered":"2024-09-12T07:22:21.148026+0000","last_unstale":"2024-09-12T07:22:48.383760+0000","last_undegraded":"2024-09-12T07:22:48.383760+0000","last_fullsized":"2024-09-12T07:22:48.383760+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'388","last_scrub_stamp":"2024-09-12T07:22:48.383662+0000","last_deep_scrub":"186'388","last_deep_scrub_stamp":"2024-09-12T07:22:48.383662+0000","last_clean_scrub_stamp":"2024-09-12T07:22:48.383662+0000","objects_scrubbed":13,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:57.683199+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0017138209999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1348,"num_read_kb":18539,"num_write":421,"num_write_kb":14413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":10121363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"186'350","reported_seq":7185,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:52.698673+0000","last_change":"2024-09-12T07:22:52.698673+0000","last_active":"2024-09-12T07:22:52.698673+0000","last_peered":"2024-09-12T07:22:52.698673+0000","last_clean":"2024-09-12T07:22:52.698673+0000","last_became_active":"2024-09-12T07:20:07.184120+0000","last_became_peered":"2024-09-12T07:20:07.184120+0000","last_unstale":"2024-09-12T07:22:52.698673+0000","last_undegraded":"2024-09-12T07:22:52.698673+0000","last_fullsized":"2024-09-12T07:22:52.698673+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'350","last_scrub_stamp":"2024-09-12T07:22:52.698572+0000","last_deep_scrub":"186'350","last_deep_scrub_stamp":"2024-09-12T07:22:52.698572+0000","last_clean_scrub_stamp":"2024-09-12T07:22:52.698572+0000","objects_scrubbed":6,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:40.102393+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00080767000000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7070,"num_read_kb":53310,"num_write":299,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4217,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"186'402","reported_seq":5869,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:43.355855+0000","last_change":"2024-09-12T07:22:43.355855+0000","last_active":"2024-09-12T07:22:43.355855+0000","last_peered":"2024-09-12T07:22:43.355855+0000","last_clean":"2024-09-12T07:22:43.355855+0000","last_became_active":"2024-09-12T07:22:27.212420+0000","last_became_peered":"2024-09-12T07:22:27.212420+0000","last_unstale":"2024-09-12T07:22:43.355855+0000","last_undegraded":"2024-09-12T07:22:43.355855+0000","last_fullsized":"2024-09-12T07:22:43.355855+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'402","last_scrub_stamp":"2024-09-12T07:22:43.355732+0000","last_deep_scrub":"186'402","last_deep_scrub_stamp":"2024-09-12T07:22:43.355732+0000","last_clean_scrub_stamp":"2024-09-12T07:22:43.355732+0000","objects_scrubbed":11,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:16:50.582545+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00097451600000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5380,"num_read_kb":60804,"num_write":390,"num_write_kb":30252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":31149476,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":149,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:45.017390+0000","last_change":"2024-09-12T07:22:45.017390+0000","last_active":"2024-09-12T07:22:45.017390+0000","last_peered":"2024-09-12T07:22:45.017390+0000","last_clean":"2024-09-12T07:22:45.017390+0000","last_became_active":"2024-09-12T07:22:31.307821+0000","last_became_peered":"2024-09-12T07:22:31.307821+0000","last_unstale":"2024-09-12T07:22:45.017390+0000","last_undegraded":"2024-09-12T07:22:45.017390+0000","last_fullsized":"2024-09-12T07:22:45.017390+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:22:45.017308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:22:45.017308+0000","last_clean_scrub_stamp":"2024-09-12T07:22:45.017308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:05.927137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"186'656","reported_seq":6291,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:41.783694+0000","last_change":"2024-09-12T07:22:41.783694+0000","last_active":"2024-09-12T07:22:41.783694+0000","last_peered":"2024-09-12T07:22:41.783694+0000","last_clean":"2024-09-12T07:22:41.783694+0000","last_became_active":"2024-09-12T07:22:21.152036+0000","last_became_peered":"2024-09-12T07:22:21.152036+0000","last_unstale":"2024-09-12T07:22:41.783694+0000","last_undegraded":"2024-09-12T07:22:41.783694+0000","last_fullsized":"2024-09-12T07:22:41.783694+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'656","last_scrub_stamp":"2024-09-12T07:22:41.783599+0000","last_deep_scrub":"186'656","last_deep_scrub_stamp":"2024-09-12T07:22:41.783599+0000","last_clean_scrub_stamp":"2024-09-12T07:22:41.783599+0000","objects_scrubbed":17,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:37:07.820257+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0014037019999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6009,"num_read_kb":23012,"num_write":704,"num_write_kb":11415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":1480654,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:43.092240+0000","last_change":"2024-09-12T07:22:43.092240+0000","last_active":"2024-09-12T07:22:43.092240+0000","last_peered":"2024-09-12T07:22:43.092240+0000","last_clean":"2024-09-12T07:22:43.092240+0000","last_became_active":"2024-09-12T07:22:18.173148+0000","last_became_peered":"2024-09-12T07:22:18.173148+0000","last_unstale":"2024-09-12T07:22:43.092240+0000","last_undegraded":"2024-09-12T07:22:43.092240+0000","last_fullsized":"2024-09-12T07:22:43.092240+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:22:43.092169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:22:43.092169+0000","last_clean_scrub_stamp":"2024-09-12T07:22:43.092169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:08.713796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"186'685","reported_seq":2223,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:44.174921+0000","last_change":"2024-09-12T07:22:44.174921+0000","last_active":"2024-09-12T07:22:44.174921+0000","last_peered":"2024-09-12T07:22:44.174921+0000","last_clean":"2024-09-12T07:22:44.174921+0000","last_became_active":"2024-09-12T07:20:07.177916+0000","last_became_peered":"2024-09-12T07:20:07.177916+0000","last_unstale":"2024-09-12T07:22:44.174921+0000","last_undegraded":"2024-09-12T07:22:44.174921+0000","last_fullsized":"2024-09-12T07:22:44.174921+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'685","last_scrub_stamp":"2024-09-12T07:22:44.174848+0000","last_deep_scrub":"186'685","last_deep_scrub_stamp":"2024-09-12T07:22:44.174848+0000","last_clean_scrub_stamp":"2024-09-12T07:22:44.174848+0000","objects_scrubbed":6,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:59.876227+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0025712959999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3057,"num_read_kb":14289,"num_write":898,"num_write_kb":11329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1247598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":356,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:46.771780+0000","last_change":"2024-09-12T07:22:46.771780+0000","last_active":"2024-09-12T07:22:46.771780+0000","last_peered":"2024-09-12T07:22:46.771780+0000","last_clean":"2024-09-12T07:22:46.771780+0000","last_became_active":"2024-09-12T07:19:30.718820+0000","last_became_peered":"2024-09-12T07:19:30.718820+0000","last_unstale":"2024-09-12T07:22:46.771780+0000","last_undegraded":"2024-09-12T07:22:46.771780+0000","last_fullsized":"2024-09-12T07:22:46.771780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:22:46.771683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:22:46.771683+0000","last_clean_scrub_stamp":"2024-09-12T07:22:46.771683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:59:26.638259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056804700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"186'515","reported_seq":3867,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:44.140622+0000","last_change":"2024-09-12T07:22:44.140622+0000","last_active":"2024-09-12T07:22:44.140622+0000","last_peered":"2024-09-12T07:22:44.140622+0000","last_clean":"2024-09-12T07:22:44.140622+0000","last_became_active":"2024-09-12T07:22:21.147703+0000","last_became_peered":"2024-09-12T07:22:21.147703+0000","last_unstale":"2024-09-12T07:22:44.140622+0000","last_undegraded":"2024-09-12T07:22:44.140622+0000","last_fullsized":"2024-09-12T07:22:44.140622+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'515","last_scrub_stamp":"2024-09-12T07:22:44.140545+0000","last_deep_scrub":"186'515","last_deep_scrub_stamp":"2024-09-12T07:22:44.140545+0000","last_clean_scrub_stamp":"2024-09-12T07:22:44.140545+0000","objects_scrubbed":9,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:38.674784+0000","scrub_duration":5,"objects_trimmed":6,"snaptrim_duration":0.023667932999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4682,"num_read_kb":14844,"num_write":579,"num_write_kb":8813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6163290,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:42.095173+0000","last_change":"2024-09-12T07:22:42.095173+0000","last_active":"2024-09-12T07:22:42.095173+0000","last_peered":"2024-09-12T07:22:42.095173+0000","last_clean":"2024-09-12T07:22:42.095173+0000","last_became_active":"2024-09-12T07:22:18.169285+0000","last_became_peered":"2024-09-12T07:22:18.169285+0000","last_unstale":"2024-09-12T07:22:42.095173+0000","last_undegraded":"2024-09-12T07:22:42.095173+0000","last_fullsized":"2024-09-12T07:22:42.095173+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:22:42.095102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:22:42.095102+0000","last_clean_scrub_stamp":"2024-09-12T07:22:42.095102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:53:43.674678+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"186'575","reported_seq":10858,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:47.409917+0000","last_change":"2024-09-12T07:22:47.409917+0000","last_active":"2024-09-12T07:22:47.409917+0000","last_peered":"2024-09-12T07:22:47.409917+0000","last_clean":"2024-09-12T07:22:47.409917+0000","last_became_active":"2024-09-12T07:22:22.151818+0000","last_became_peered":"2024-09-12T07:22:22.151818+0000","last_unstale":"2024-09-12T07:22:47.409917+0000","last_undegraded":"2024-09-12T07:22:47.409917+0000","last_fullsized":"2024-09-12T07:22:47.409917+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'575","last_scrub_stamp":"2024-09-12T07:22:47.409820+0000","last_deep_scrub":"186'575","last_deep_scrub_stamp":"2024-09-12T07:22:47.409820+0000","last_clean_scrub_stamp":"2024-09-12T07:22:47.409820+0000","objects_scrubbed":10,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:11.823319+0000","scrub_duration":6,"objects_trimmed":18,"snaptrim_duration":0.023276249999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":11624,"num_read_kb":77562,"num_write":595,"num_write_kb":25411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":22089334,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"186'521","reported_seq":4521,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:44.381948+0000","last_change":"2024-09-12T07:22:44.381948+0000","last_active":"2024-09-12T07:22:44.381948+0000","last_peered":"2024-09-12T07:22:44.381948+0000","last_clean":"2024-09-12T07:22:44.381948+0000","last_became_active":"2024-09-12T07:22:27.213073+0000","last_became_peered":"2024-09-12T07:22:27.213073+0000","last_unstale":"2024-09-12T07:22:44.381948+0000","last_undegraded":"2024-09-12T07:22:44.381948+0000","last_fullsized":"2024-09-12T07:22:44.381948+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'521","last_scrub_stamp":"2024-09-12T07:22:44.381834+0000","last_deep_scrub":"186'521","last_deep_scrub_stamp":"2024-09-12T07:22:44.381834+0000","last_clean_scrub_stamp":"2024-09-12T07:22:44.381834+0000","objects_scrubbed":18,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:38.725206+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0018780750000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":3159,"num_read_kb":12201,"num_write":460,"num_write_kb":10579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":2603172,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"186'3","reported_seq":57,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:45.411717+0000","last_change":"2024-09-12T07:22:45.411717+0000","last_active":"2024-09-12T07:22:45.411717+0000","last_peered":"2024-09-12T07:22:45.411717+0000","last_clean":"2024-09-12T07:22:45.411717+0000","last_became_active":"2024-09-12T07:22:19.178595+0000","last_became_peered":"2024-09-12T07:22:19.178595+0000","last_unstale":"2024-09-12T07:22:45.411717+0000","last_undegraded":"2024-09-12T07:22:45.411717+0000","last_fullsized":"2024-09-12T07:22:45.411717+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'3","last_scrub_stamp":"2024-09-12T07:22:45.411626+0000","last_deep_scrub":"186'3","last_deep_scrub_stamp":"2024-09-12T07:22:45.411626+0000","last_clean_scrub_stamp":"2024-09-12T07:22:45.411626+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:20:36.178712+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0013169200000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"186'799","reported_seq":3351,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:45.163123+0000","last_change":"2024-09-12T07:22:45.163123+0000","last_active":"2024-09-12T07:22:45.163123+0000","last_peered":"2024-09-12T07:22:45.163123+0000","last_clean":"2024-09-12T07:22:45.163123+0000","last_became_active":"2024-09-12T07:22:21.147758+0000","last_became_peered":"2024-09-12T07:22:21.147758+0000","last_unstale":"2024-09-12T07:22:45.163123+0000","last_undegraded":"2024-09-12T07:22:45.163123+0000","last_fullsized":"2024-09-12T07:22:45.163123+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'799","last_scrub_stamp":"2024-09-12T07:22:45.163030+0000","last_deep_scrub":"186'799","last_deep_scrub_stamp":"2024-09-12T07:22:45.163030+0000","last_clean_scrub_stamp":"2024-09-12T07:22:45.163030+0000","objects_scrubbed":18,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:37.699271+0000","scrub_duration":13,"objects_trimmed":10,"snaptrim_duration":0.033858052999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5450,"num_read_kb":11647,"num_write":1052,"num_write_kb":13386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":9483314,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":307,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:43.076810+0000","last_change":"2024-09-12T07:22:43.076810+0000","last_active":"2024-09-12T07:22:43.076810+0000","last_peered":"2024-09-12T07:22:43.076810+0000","last_clean":"2024-09-12T07:22:43.076810+0000","last_became_active":"2024-09-12T07:22:09.182547+0000","last_became_peered":"2024-09-12T07:22:09.182547+0000","last_unstale":"2024-09-12T07:22:43.076810+0000","last_undegraded":"2024-09-12T07:22:43.076810+0000","last_fullsized":"2024-09-12T07:22:43.076810+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:22:43.076748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:22:43.076748+0000","last_clean_scrub_stamp":"2024-09-12T07:22:43.076748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:42:56.457859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037458199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"186'413","reported_seq":10560,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:43.838012+0000","last_change":"2024-09-12T07:22:43.838012+0000","last_active":"2024-09-12T07:22:43.838012+0000","last_peered":"2024-09-12T07:22:43.838012+0000","last_clean":"2024-09-12T07:22:43.838012+0000","last_became_active":"2024-09-12T07:22:01.146421+0000","last_became_peered":"2024-09-12T07:22:01.146421+0000","last_unstale":"2024-09-12T07:22:43.838012+0000","last_undegraded":"2024-09-12T07:22:43.838012+0000","last_fullsized":"2024-09-12T07:22:43.838012+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'413","last_scrub_stamp":"2024-09-12T07:22:43.837908+0000","last_deep_scrub":"186'413","last_deep_scrub_stamp":"2024-09-12T07:22:43.837908+0000","last_clean_scrub_stamp":"2024-09-12T07:22:43.837908+0000","objects_scrubbed":7,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:48.764681+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00094442799999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11174,"num_read_kb":59795,"num_write":376,"num_write_kb":27716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":22543830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":368,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:45.801852+0000","last_change":"2024-09-12T07:22:45.801852+0000","last_active":"2024-09-12T07:22:45.801852+0000","last_peered":"2024-09-12T07:22:45.801852+0000","last_clean":"2024-09-12T07:22:45.801852+0000","last_became_active":"2024-09-12T07:22:18.171266+0000","last_became_peered":"2024-09-12T07:22:18.171266+0000","last_unstale":"2024-09-12T07:22:45.801852+0000","last_undegraded":"2024-09-12T07:22:45.801852+0000","last_fullsized":"2024-09-12T07:22:45.801852+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T07:22:45.801749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T07:22:45.801749+0000","last_clean_scrub_stamp":"2024-09-12T07:22:45.801749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:33.319998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054085099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"186'1073","reported_seq":6964,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:47.108736+0000","last_change":"2024-09-12T07:22:47.108736+0000","last_active":"2024-09-12T07:22:47.108736+0000","last_peered":"2024-09-12T07:22:47.108736+0000","last_clean":"2024-09-12T07:22:47.108736+0000","last_became_active":"2024-09-12T07:22:22.150316+0000","last_became_peered":"2024-09-12T07:22:22.150316+0000","last_unstale":"2024-09-12T07:22:47.108736+0000","last_undegraded":"2024-09-12T07:22:47.108736+0000","last_fullsized":"2024-09-12T07:22:47.108736+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'1073","last_scrub_stamp":"2024-09-12T07:22:47.108637+0000","last_deep_scrub":"186'1073","last_deep_scrub_stamp":"2024-09-12T07:22:47.108637+0000","last_clean_scrub_stamp":"2024-09-12T07:22:47.108637+0000","objects_scrubbed":11,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:04.288560+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0021189680000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8933,"num_read_kb":59814,"num_write":1570,"num_write_kb":27462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":14379761,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"186'2","reported_seq":54,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:44.431081+0000","last_change":"2024-09-12T07:22:44.431081+0000","last_active":"2024-09-12T07:22:44.431081+0000","last_peered":"2024-09-12T07:22:44.431081+0000","last_clean":"2024-09-12T07:22:44.431081+0000","last_became_active":"2024-09-12T07:22:19.170715+0000","last_became_peered":"2024-09-12T07:22:19.170715+0000","last_unstale":"2024-09-12T07:22:44.431081+0000","last_undegraded":"2024-09-12T07:22:44.431081+0000","last_fullsized":"2024-09-12T07:22:44.431081+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'2","last_scrub_stamp":"2024-09-12T07:22:44.430995+0000","last_deep_scrub":"186'2","last_deep_scrub_stamp":"2024-09-12T07:22:44.430995+0000","last_clean_scrub_stamp":"2024-09-12T07:22:44.430995+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:03.846379+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"186'74","reported_seq":449,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:44.777770+0000","last_change":"2024-09-12T07:22:44.777770+0000","last_active":"2024-09-12T07:22:44.777770+0000","last_peered":"2024-09-12T07:22:44.777770+0000","last_clean":"2024-09-12T07:22:44.777770+0000","last_became_active":"2024-09-12T07:19:25.192566+0000","last_became_peered":"2024-09-12T07:19:25.192566+0000","last_unstale":"2024-09-12T07:22:44.777770+0000","last_undegraded":"2024-09-12T07:22:44.777770+0000","last_fullsized":"2024-09-12T07:22:44.777770+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'74","last_scrub_stamp":"2024-09-12T07:22:44.777666+0000","last_deep_scrub":"186'74","last_deep_scrub_stamp":"2024-09-12T07:22:44.777666+0000","last_clean_scrub_stamp":"2024-09-12T07:22:44.777666+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:00.006021+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"186'971","reported_seq":3124,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:43.139726+0000","last_change":"2024-09-12T07:22:43.139726+0000","last_active":"2024-09-12T07:22:43.139726+0000","last_peered":"2024-09-12T07:22:43.139726+0000","last_clean":"2024-09-12T07:22:43.139726+0000","last_became_active":"2024-09-12T07:22:21.150083+0000","last_became_peered":"2024-09-12T07:22:21.150083+0000","last_unstale":"2024-09-12T07:22:43.139726+0000","last_undegraded":"2024-09-12T07:22:43.139726+0000","last_fullsized":"2024-09-12T07:22:43.139726+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'971","last_scrub_stamp":"2024-09-12T07:22:43.139647+0000","last_deep_scrub":"186'971","last_deep_scrub_stamp":"2024-09-12T07:22:43.139647+0000","last_clean_scrub_stamp":"2024-09-12T07:22:43.139647+0000","objects_scrubbed":9,"log_size":971,"log_dups_size":0,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:41.217748+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0011038020000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5044,"num_read_kb":23221,"num_write":1448,"num_write_kb":24489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":21462395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"186'564","reported_seq":5449,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:45.417989+0000","last_change":"2024-09-12T07:22:45.417989+0000","last_active":"2024-09-12T07:22:45.417989+0000","last_peered":"2024-09-12T07:22:45.417989+0000","last_clean":"2024-09-12T07:22:45.417989+0000","last_became_active":"2024-09-12T07:22:27.212942+0000","last_became_peered":"2024-09-12T07:22:27.212942+0000","last_unstale":"2024-09-12T07:22:45.417989+0000","last_undegraded":"2024-09-12T07:22:45.417989+0000","last_fullsized":"2024-09-12T07:22:45.417989+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'564","last_scrub_stamp":"2024-09-12T07:22:45.417883+0000","last_deep_scrub":"186'564","last_deep_scrub_stamp":"2024-09-12T07:22:45.417883+0000","last_clean_scrub_stamp":"2024-09-12T07:22:45.417883+0000","objects_scrubbed":15,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:01.456416+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0021244950000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7563,"num_read_kb":32357,"num_write":525,"num_write_kb":20455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1710555,"num_keys_recovered":41,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"186'553","reported_seq":6698,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:46.376832+0000","last_change":"2024-09-12T07:22:46.376832+0000","last_active":"2024-09-12T07:22:46.376832+0000","last_peered":"2024-09-12T07:22:46.376832+0000","last_clean":"2024-09-12T07:22:46.376832+0000","last_became_active":"2024-09-12T07:22:27.213533+0000","last_became_peered":"2024-09-12T07:22:27.213533+0000","last_unstale":"2024-09-12T07:22:46.376832+0000","last_undegraded":"2024-09-12T07:22:46.376832+0000","last_fullsized":"2024-09-12T07:22:46.376832+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'553","last_scrub_stamp":"2024-09-12T07:22:46.376732+0000","last_deep_scrub":"186'553","last_deep_scrub_stamp":"2024-09-12T07:22:46.376732+0000","last_clean_scrub_stamp":"2024-09-12T07:22:46.376732+0000","objects_scrubbed":13,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:08:39.857513+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00069883599999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":6247,"num_read_kb":49000,"num_write":522,"num_write_kb":25535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":42565291,"num_keys_recovered":104,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"186'702","reported_seq":3585,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:45.128924+0000","last_change":"2024-09-12T07:22:45.128924+0000","last_active":"2024-09-12T07:22:45.128924+0000","last_peered":"2024-09-12T07:22:45.128924+0000","last_clean":"2024-09-12T07:22:45.128924+0000","last_became_active":"2024-09-12T07:20:07.180929+0000","last_became_peered":"2024-09-12T07:20:07.180929+0000","last_unstale":"2024-09-12T07:22:45.128924+0000","last_undegraded":"2024-09-12T07:22:45.128924+0000","last_fullsized":"2024-09-12T07:22:45.128924+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'702","last_scrub_stamp":"2024-09-12T07:22:45.128836+0000","last_deep_scrub":"186'702","last_deep_scrub_stamp":"2024-09-12T07:22:45.128836+0000","last_clean_scrub_stamp":"2024-09-12T07:22:45.128836+0000","objects_scrubbed":10,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:23.907117+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0015261610000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6009,"num_read_kb":21069,"num_write":727,"num_write_kb":17071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9954635,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"186'453","reported_seq":9938,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:46.427244+0000","last_change":"2024-09-12T07:22:46.427244+0000","last_active":"2024-09-12T07:22:46.427244+0000","last_peered":"2024-09-12T07:22:46.427244+0000","last_clean":"2024-09-12T07:22:46.427244+0000","last_became_active":"2024-09-12T07:22:22.149715+0000","last_became_peered":"2024-09-12T07:22:22.149715+0000","last_unstale":"2024-09-12T07:22:46.427244+0000","last_undegraded":"2024-09-12T07:22:46.427244+0000","last_fullsized":"2024-09-12T07:22:46.427244+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'453","last_scrub_stamp":"2024-09-12T07:22:46.427151+0000","last_deep_scrub":"186'453","last_deep_scrub_stamp":"2024-09-12T07:22:46.427151+0000","last_clean_scrub_stamp":"2024-09-12T07:22:46.427151+0000","objects_scrubbed":8,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:07.084800+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0016506240000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11019,"num_read_kb":49685,"num_write":382,"num_write_kb":15924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":18973449,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"186'781","reported_seq":3660,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:47.743188+0000","last_change":"2024-09-12T07:22:47.743188+0000","last_active":"2024-09-12T07:22:47.743188+0000","last_peered":"2024-09-12T07:22:47.743188+0000","last_clean":"2024-09-12T07:22:47.743188+0000","last_became_active":"2024-09-12T07:22:22.152455+0000","last_became_peered":"2024-09-12T07:22:22.152455+0000","last_unstale":"2024-09-12T07:22:47.743188+0000","last_undegraded":"2024-09-12T07:22:47.743188+0000","last_fullsized":"2024-09-12T07:22:47.743188+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'781","last_scrub_stamp":"2024-09-12T07:22:47.743092+0000","last_deep_scrub":"186'781","last_deep_scrub_stamp":"2024-09-12T07:22:47.743092+0000","last_clean_scrub_stamp":"2024-09-12T07:22:47.743092+0000","objects_scrubbed":12,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:36.030907+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0014515540000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5118,"num_read_kb":30709,"num_write":1176,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":7370075,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"186'541","reported_seq":5427,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:48.724958+0000","last_change":"2024-09-12T07:22:48.724958+0000","last_active":"2024-09-12T07:22:48.724958+0000","last_peered":"2024-09-12T07:22:48.724958+0000","last_clean":"2024-09-12T07:22:48.724958+0000","last_became_active":"2024-09-12T07:22:00.145108+0000","last_became_peered":"2024-09-12T07:22:00.145108+0000","last_unstale":"2024-09-12T07:22:48.724958+0000","last_undegraded":"2024-09-12T07:22:48.724958+0000","last_fullsized":"2024-09-12T07:22:48.724958+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'541","last_scrub_stamp":"2024-09-12T07:22:48.724857+0000","last_deep_scrub":"186'541","last_deep_scrub_stamp":"2024-09-12T07:22:48.724857+0000","last_clean_scrub_stamp":"2024-09-12T07:22:48.724857+0000","objects_scrubbed":10,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:52:50.800393+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0010801299999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6080,"num_read_kb":38867,"num_write":581,"num_write_kb":16891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":10508628,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"186'603","reported_seq":3038,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:49.688493+0000","last_change":"2024-09-12T07:22:49.688493+0000","last_active":"2024-09-12T07:22:49.688493+0000","last_peered":"2024-09-12T07:22:49.688493+0000","last_clean":"2024-09-12T07:22:49.688493+0000","last_became_active":"2024-09-12T07:22:01.143359+0000","last_became_peered":"2024-09-12T07:22:01.143359+0000","last_unstale":"2024-09-12T07:22:49.688493+0000","last_undegraded":"2024-09-12T07:22:49.688493+0000","last_fullsized":"2024-09-12T07:22:49.688493+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'603","last_scrub_stamp":"2024-09-12T07:22:49.688384+0000","last_deep_scrub":"186'603","last_deep_scrub_stamp":"2024-09-12T07:22:49.688384+0000","last_clean_scrub_stamp":"2024-09-12T07:22:49.688384+0000","objects_scrubbed":7,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.149847+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001515553,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3463,"num_read_kb":13505,"num_write":637,"num_write_kb":16522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":8725508,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"186'350","reported_seq":1817,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:43.997863+0000","last_change":"2024-09-12T07:22:43.997863+0000","last_active":"2024-09-12T07:22:43.997863+0000","last_peered":"2024-09-12T07:22:43.997863+0000","last_clean":"2024-09-12T07:22:43.997863+0000","last_became_active":"2024-09-12T07:22:31.245905+0000","last_became_peered":"2024-09-12T07:22:31.245905+0000","last_unstale":"2024-09-12T07:22:43.997863+0000","last_undegraded":"2024-09-12T07:22:43.997863+0000","last_fullsized":"2024-09-12T07:22:43.997863+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'350","last_scrub_stamp":"2024-09-12T07:22:43.997741+0000","last_deep_scrub":"186'350","last_deep_scrub_stamp":"2024-09-12T07:22:43.997741+0000","last_clean_scrub_stamp":"2024-09-12T07:22:43.997741+0000","objects_scrubbed":12,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:39:00.209537+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0010816669999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1472,"num_read_kb":27101,"num_write":327,"num_write_kb":18963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"186'461","reported_seq":7395,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:51.719410+0000","last_change":"2024-09-12T07:22:51.719410+0000","last_active":"2024-09-12T07:22:51.719410+0000","last_peered":"2024-09-12T07:22:51.719410+0000","last_clean":"2024-09-12T07:22:51.719410+0000","last_became_active":"2024-09-12T07:22:01.143631+0000","last_became_peered":"2024-09-12T07:22:01.143631+0000","last_unstale":"2024-09-12T07:22:51.719410+0000","last_undegraded":"2024-09-12T07:22:51.719410+0000","last_fullsized":"2024-09-12T07:22:51.719410+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'461","last_scrub_stamp":"2024-09-12T07:22:51.719309+0000","last_deep_scrub":"186'461","last_deep_scrub_stamp":"2024-09-12T07:22:51.719309+0000","last_clean_scrub_stamp":"2024-09-12T07:22:51.719309+0000","objects_scrubbed":7,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:30.561559+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00065451099999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7090,"num_read_kb":47369,"num_write":461,"num_write_kb":21538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":17766842,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"186'413","reported_seq":11223,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:44.877189+0000","last_change":"2024-09-12T07:22:44.877189+0000","last_active":"2024-09-12T07:22:44.877189+0000","last_peered":"2024-09-12T07:22:44.877189+0000","last_clean":"2024-09-12T07:22:44.877189+0000","last_became_active":"2024-09-12T07:19:42.450165+0000","last_became_peered":"2024-09-12T07:19:42.450165+0000","last_unstale":"2024-09-12T07:22:44.877189+0000","last_undegraded":"2024-09-12T07:22:44.877189+0000","last_fullsized":"2024-09-12T07:22:44.877189+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'413","last_scrub_stamp":"2024-09-12T07:22:44.877101+0000","last_deep_scrub":"186'413","last_deep_scrub_stamp":"2024-09-12T07:22:44.877101+0000","last_clean_scrub_stamp":"2024-09-12T07:22:44.877101+0000","objects_scrubbed":5,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:23.555767+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0019415999999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11863,"num_read_kb":47954,"num_write":410,"num_write_kb":15586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"186'484","reported_seq":19744,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-12T07:22:50.675688+0000","last_change":"2024-09-12T07:22:50.675688+0000","last_active":"2024-09-12T07:22:50.675688+0000","last_peered":"2024-09-12T07:22:50.675688+0000","last_clean":"2024-09-12T07:22:50.675688+0000","last_became_active":"2024-09-12T07:22:01.143577+0000","last_became_peered":"2024-09-12T07:22:01.143577+0000","last_unstale":"2024-09-12T07:22:50.675688+0000","last_undegraded":"2024-09-12T07:22:50.675688+0000","last_fullsized":"2024-09-12T07:22:50.675688+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'484","last_scrub_stamp":"2024-09-12T07:22:50.675587+0000","last_deep_scrub":"186'484","last_deep_scrub_stamp":"2024-09-12T07:22:50.675587+0000","last_clean_scrub_stamp":"2024-09-12T07:22:50.675587+0000","objects_scrubbed":8,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:12.344378+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0010515870000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":18947,"num_read_kb":82796,"num_write":553,"num_write_kb":44274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":53090062,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":323,"num_object_clones":0,"num_object_copies":646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":319,"num_read":260585,"num_read_kb":1190964,"num_write":20483,"num_write_kb":597347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5845,"num_bytes_recovered":432794818,"num_keys_recovered":767,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},"log_size":18282,"ondisk_log_size":18282,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60320,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311520,"statfs":{"total":96636764160,"available":96574996480,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33372,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338468,"statfs":{"total":96636764160,"available":96602591232,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542192,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55584,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316256,"statfs":{"total":96636764160,"available":96579846144,"internally_reserved":0,"allocated":2244608,"data_stored":1081409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51424,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320416,"statfs":{"total":96636764160,"available":96584105984,"internally_reserved":0,"allocated":2244608,"data_stored":1081409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69468,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302372,"statfs":{"total":96636764160,"available":96565628928,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":165,"seq":708669603853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31004,"kb_used_data":2188,"kb_used_omap":23,"kb_used_meta":28776,"kb_avail":94340836,"statfs":{"total":96636764160,"available":96605016064,"internally_reserved":0,"allocated":2240512,"data_stored":1081390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24213,"internal_metadata":29466987},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542193,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55588,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316252,"statfs":{"total":96636764160,"available":96579842048,"internally_reserved":0,"allocated":2248704,"data_stored":1081428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47840,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324000,"statfs":{"total":96636764160,"available":96587776000,"internally_reserved":0,"allocated":2834432,"data_stored":1671758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12604,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T07:23:05.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-12T07:23:05.362 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-12T07:23:05.362 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-12T07:23:05.362 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-12T07:23:05.362 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:05.401 INFO:tasks.ceph.osd.0:Stopped 2024-09-12T07:23:05.401 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-12T07:23:05.401 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:05.480 INFO:tasks.ceph.osd.1:Stopped 2024-09-12T07:23:05.481 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-12T07:23:05.481 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:05.549 INFO:tasks.ceph.osd.2:Stopped 2024-09-12T07:23:05.549 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-12T07:23:05.549 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:05.624 INFO:tasks.ceph.osd.3:Stopped 2024-09-12T07:23:05.624 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-12T07:23:05.624 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:05.662 INFO:tasks.ceph.osd.4:Stopped 2024-09-12T07:23:05.662 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-12T07:23:05.662 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:05.740 INFO:tasks.ceph.osd.5:Stopped 2024-09-12T07:23:05.740 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-12T07:23:05.740 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:05.817 INFO:tasks.ceph.osd.6:Stopped 2024-09-12T07:23:05.817 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-12T07:23:05.817 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:05.895 INFO:tasks.ceph.osd.7:Stopped 2024-09-12T07:23:05.896 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-12T07:23:05.896 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-12T07:23:05.896 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:05.922 INFO:tasks.ceph.mgr.y:Stopped 2024-09-12T07:23:05.922 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-12T07:23:05.922 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:05.966 INFO:tasks.ceph.mgr.x:Stopped 2024-09-12T07:23:05.966 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-12T07:23:05.966 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-12T07:23:05.966 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:05.978 INFO:tasks.ceph.mon.a:Stopped 2024-09-12T07:23:05.978 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-12T07:23:05.978 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:06.030 INFO:tasks.ceph.mon.c:Stopped 2024-09-12T07:23:06.030 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-12T07:23:06.030 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T07:23:06.045 INFO:tasks.ceph.mon.b:Stopped 2024-09-12T07:23:06.045 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-12T07:23:06.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-12T07:23:06.109 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T07:23:06.109 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-12T07:23:06.334 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T07:23:06.334 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-12T07:23:06.426 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T07:23:06.426 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-12T07:23:06.483 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T07:23:06.483 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-12T07:23:06.525 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T07:23:06.525 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-12T07:23:06.657 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T07:23:06.657 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-12T07:23:06.745 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T07:23:06.745 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-12T07:23:06.802 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T07:23:06.802 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-12T07:23:06.860 INFO:tasks.ceph:Archiving mon data... 2024-09-12T07:23:06.862 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413/data/mon.a.tgz 2024-09-12T07:23:06.862 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-09-12T07:23:06.879 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.fZX4NOnMqX 2024-09-12T07:23:06.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.fZX4NOnMqX 2024-09-12T07:23:07.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.fZX4NOnMqX 2024-09-12T07:23:07.134 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.fZX4NOnMqX is 418KB 2024-09-12T07:23:07.233 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.fZX4NOnMqX 2024-09-12T07:23:07.248 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413/data/mon.c.tgz 2024-09-12T07:23:07.248 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-09-12T07:23:07.303 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.pJ6eaoBCJb 2024-09-12T07:23:07.303 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.pJ6eaoBCJb 2024-09-12T07:23:07.492 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.pJ6eaoBCJb 2024-09-12T07:23:07.576 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.pJ6eaoBCJb is 570KB 2024-09-12T07:23:07.675 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.pJ6eaoBCJb 2024-09-12T07:23:07.691 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413/data/mon.b.tgz 2024-09-12T07:23:07.691 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-09-12T07:23:07.711 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.0x487YdpXO 2024-09-12T07:23:07.711 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.0x487YdpXO 2024-09-12T07:23:07.920 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.0x487YdpXO 2024-09-12T07:23:08.006 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.0x487YdpXO is 570KB 2024-09-12T07:23:08.108 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.0x487YdpXO 2024-09-12T07:23:08.124 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-12T07:23:08.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T07:23:08.127 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T07:23:08.208 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-12T07:23:08.246 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-12T07:23:08.247 INFO:tasks.ceph:Archiving crash dumps... 2024-09-12T07:23:08.250 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413/remote/smithi046/crash 2024-09-12T07:23:08.254 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-12T07:23:08.280 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413/remote/smithi064/crash 2024-09-12T07:23:08.281 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-12T07:23:08.311 INFO:tasks.ceph:Compressing logs... 2024-09-12T07:23:08.311 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T07:23:08.326 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T07:23:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-12T07:23:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50099.log 2024-09-12T07:23:08.396 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33504.log 2024-09-12T07:23:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-12T07:23:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33744.log 2024-09-12T07:23:08.397 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33504.log.gz 2024-09-12T07:23:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-12T07:23:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33744.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-12T07:23:08.397 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33744.log.gz 2024-09-12T07:23:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33448.log 2024-09-12T07:23:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33544.log 2024-09-12T07:23:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33448.log.gz 2024-09-12T07:23:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33624.log 2024-09-12T07:23:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33544.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33544.log.gz 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33624.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33624.log.gz 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33584.log 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33704.log 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33584.log.gz 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33664.log 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr: 85.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33664.log: /var/log/ceph/ceph-client.admin.33704.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33664.log.gz 2024-09-12T07:23:08.398 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33704.log.gz 2024-09-12T07:23:08.439 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mon.b.log: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-12T07:23:08.482 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61422.log 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.50099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63606.log 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50099.log.gz 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61380.log 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61422.log.gz 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60745.log 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63606.log.gz 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61296.log 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61380.log.gz 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60960.log 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60745.log.gz 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54506.log 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61296.log.gz 2024-09-12T07:23:08.483 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38526.log 2024-09-12T07:23:08.484 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60960.log.gz 2024-09-12T07:23:08.484 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47390.log 2024-09-12T07:23:08.484 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54506.log.gz 2024-09-12T07:23:08.484 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38652.log 2024-09-12T07:23:08.484 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38526.log.gz 2024-09-12T07:23:08.484 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60599.log 2024-09-12T07:23:08.484 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47390.log.gz 2024-09-12T07:23:08.484 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62057.log 2024-09-12T07:23:08.484 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38652.log.gz 2024-09-12T07:23:08.484 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44101.log 2024-09-12T07:23:08.484 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60599.log.gz 2024-09-12T07:23:08.484 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47025.log 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62057.log.gz 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40122.log 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44101.log.gz 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44888.log 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47025.log.gz 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40122.log.gz 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62322.log 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45229.log 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44888.log.gz 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42813.log 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62322.log.gz 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39170.log 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45229.log.gz 2024-09-12T07:23:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46377.log 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42813.log.gz 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41093.log 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39170.log.gz 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39536.log 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46377.log.gz 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61464.log 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41093.log.gz 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48811.log 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39536.log.gz 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52252.log 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61464.log.gz 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58984.log 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48811.log.gz 2024-09-12T07:23:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45820.log 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52252.log.gz 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50465.log 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58984.log.gz 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45820.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.57134.log 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45820.log.gz 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61170.log 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50465.log.gz 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/objectstore_tool.57134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40164.log 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60918.log 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61170.log.gz 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62682.log 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40164.log.gz 2024-09-12T07:23:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56509.log 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60918.log.gz 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62682.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62682.log.gz 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61506.log 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56509.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56509.log.gz 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47969.log 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.61506.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.42691.log 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61506.log.gz 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46842.log 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47969.log.gz 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39162.log 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42691.log.gz 2024-09-12T07:23:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50659.log 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr: 25.4% -- replaced with /var/log/ceph/ceph-client.admin.46842.log.gz 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38833.log 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39162.log.gz 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58869.log 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50659.log.gz 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55324.log 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38833.log.gz 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38830.log 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58869.log.gz 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55324.log.gz 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52170.log 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.57134.log.gz 2024-09-12T07:23:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52510.log 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38830.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38830.log.gz 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52550.log 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52170.log.gz 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41768.log 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52510.log.gz 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41986.log 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52550.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52550.log.gz 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59508.log 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41768.log.gz 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54984.log 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41986.log.gz 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53486.log 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59508.log.gz 2024-09-12T07:23:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39534.log 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54984.log.gz 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53486.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.46373.log -- replaced with /var/log/ceph/ceph-client.admin.53486.log.gz 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61548.log 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39534.log.gz 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62066.log 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46373.log.gz 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61548.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61548.log.gz 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49860.log 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63564.log 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62066.log.gz 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49860.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.49860.log.gz 2024-09-12T07:23:08.491 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45459.log 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54349.log 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63564.log.gz 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63354.log 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45459.log.gz 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53749.log 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54349.log.gz 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55542.log 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63354.log.gz 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48892.log 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53749.log.gz 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45539.log 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55542.log.gz 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62281.log 2024-09-12T07:23:08.492 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48892.log.gz 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62766.log 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45539.log.gz 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61338.log 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62281.log.gz 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51065.log 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62766.log.gz 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62724.log 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61338.log.gz 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44262.log 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51065.log.gz 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62724.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.62724.log.gz 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38568.log 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53028.log 2024-09-12T07:23:08.493 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44262.log.gz 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38568.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38568.log.gz -5 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50001.log 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41589.log 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53028.log.gz 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50001.log.gz 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51743.log 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48052.log 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41589.log.gz 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51743.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51743.log.gz 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53667.log 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43017.log 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48052.log.gz 2024-09-12T07:23:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53667.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.53667.log.gz --verbose 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.38821.log 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46662.log 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43017.log.gz 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38821.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.38821.log.gz -- 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.63312.log 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39426.log 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46662.log.gz 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63312.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63312.log.gz 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62321.log 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62403.log 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39426.log.gz 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62321.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.62321.log.gz 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42524.log 2024-09-12T07:23:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38822.log 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62403.log.gz 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42524.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42524.log.gz -5 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39702.log 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38823.log 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38822.log.gz 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39702.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39702.log.gz 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40248.log 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38823.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38823.log.gz 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40248.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.40248.log.gz 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51765.log 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42527.log 2024-09-12T07:23:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45131.log 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51765.log.gz 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42527.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42527.log.gz 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39841.log 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62063.log 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45131.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45131.log.gz 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39841.log.gz 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58118.log 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46841.log 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62063.log.gz 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58118.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.58118.log.gz 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58671.log 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38394.log 2024-09-12T07:23:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46841.log.gz 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58671.log.gz 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40490.log 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46461.log 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38394.log.gz 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46258.log 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40490.log.gz 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63018.log 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46461.log.gz 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46258.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.46258.log.gz -5 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49394.log 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63144.log 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63018.log.gz 2024-09-12T07:23:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45312.log 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49394.log.gz 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54785.log 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63144.log.gz 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62406.log 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45312.log.gz 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59240.log 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54785.log.gz 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38826.log 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62406.log.gz 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63646.log 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59240.log.gz 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39742.log 0.0% 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38826.log.gz 2024-09-12T07:23:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53008.log 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63646.log.gz 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51649.log 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39742.log.gz 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53008.log.gz 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51223.log 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53870.log 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51649.log.gz 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51223.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.51223.log.gz -- 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.47851.log 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60703.log 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53870.log.gz 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47330.log 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47851.log.gz 2024-09-12T07:23:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62808.log 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60703.log.gz 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47330.log.gz 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53446.log 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43281.log 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62808.log.gz 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53446.log.gz 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39533.log 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39168.log 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43281.log.gz 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42631.log 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39533.log.gz 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33629.log 2024-09-12T07:23:08.501 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39168.log.gz 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40831.log 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42631.log.gz 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43801.log 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33629.log.gz 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40831.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40831.log.gz 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40352.log 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52669.log 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43801.log.gz 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40352.log.gz 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45249.log 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44446.log 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52669.log.gz 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45249.log.gz 2024-09-12T07:23:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48712.log 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39164.log 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44446.log.gz 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48712.log.gz 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57653.log 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58730.log 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39164.log.gz 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.57653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57653.log.gz 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.55597.log 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38305.log 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58730.log.gz 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/objectstore_tool.55597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50785.log 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60834.log 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38305.log.gz 2024-09-12T07:23:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61726.log 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50785.log.gz 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60834.log.gz 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62401.log 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27489.log 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61726.log: 94.5% -- replaced with /var/log/ceph/objectstore_tool.55597.log.gz 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61726.log.gz 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55146.log 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62401.log.gz 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.tmp-client.admin.27489.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27489.log.gz 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39366.log 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55146.log.gz 2024-09-12T07:23:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38610.log 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39366.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.39366.log.gz --verbose 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.62065.log 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62976.log 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38610.log.gz 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42591.log 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62065.log.gz 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40038.log 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62976.log.gz 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48211.log 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42591.log.gz 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.40038.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39781.log 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40038.log.gz 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54647.log 2024-09-12T07:23:08.505 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48211.log.gz 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48414.log 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39781.log.gz 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43704.log 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54647.log.gz 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48414.log.gz 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61086.log 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48091.log 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43704.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43704.log.gz 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61724.log 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61086.log.gz 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42793.log 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48091.log.gz 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61946.log 2024-09-12T07:23:08.506 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61724.log.gz 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40492.log 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42793.log.gz 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43198.log 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61946.log.gz 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42126.log 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40492.log.gz 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47512.log 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43198.log.gz 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54227.log 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42126.log.gz 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44747.log 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47512.log.gz 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47207.log 2024-09-12T07:23:08.507 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54227.log.gz 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44747.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44747.log.gz -5 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47908.log 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51263.log 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47207.log.gz 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53585.log 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47908.log.gz 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61725.log 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51263.log.gz 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63102.log 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53585.log.gz 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61725.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.46663.log -- replaced with /var/log/ceph/ceph-client.admin.61725.log.gz 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-12T07:23:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63102.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63102.log.gz 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46663.log.gz 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44282.log 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61128.log 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.44282.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44282.log.gz -5 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55203.log 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49214.log 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61128.log.gz 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61212.log 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55203.log.gz 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50842.log 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49214.log.gz 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42425.log 2024-09-12T07:23:08.509 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61212.log.gz 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40888.log 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50842.log.gz 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60876.log 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42425.log.gz 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62934.log 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40888.log.gz 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51703.log 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60876.log.gz 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62064.log 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62934.log.gz 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52806.log 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51703.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51703.log.gz 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47459.log 2024-09-12T07:23:08.510 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62064.log.gz 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63228.log 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52806.log.gz 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.47459.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.62282.log 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47459.log.gz 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63228.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63228.log.gz 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40489.log 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62282.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62282.log.gz 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58454.log 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40489.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40489.log.gz 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38450.log 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58454.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58454.log.gz 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48652.log 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49920.log 2024-09-12T07:23:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38450.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38450.log.gz 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49395.log 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48652.log.gz 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49030.log 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49920.log.gz 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46176.log 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49395.log.gz 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49030.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49030.log.gz 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43939.log 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46176.log: /var/log/ceph/ceph-osd.2.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46176.log.gz 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41250.log 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43939.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43939.log.gz 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48373.log 2024-09-12T07:23:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41250.log.gz 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61723.log 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48373.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48373.log.gz 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49619.log 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61723.log.gz 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49619.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49619.log.gz 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53933.log 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40206.log 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53933.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59106.log 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53933.log.gz 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40206.log.gz 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51973.log 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59106.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49275.log 2024-09-12T07:23:08.513 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61943.log 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51973.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63060.log 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51973.log.gz 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49275.log.gz 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61943.log.gz 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43397.log 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59438.log 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63060.log.gz 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43397.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47532.log 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43397.log.gz 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59438.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59438.log.gz 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr: 92.9%gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-12T07:23:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44142.log 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47532.log.gz 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58393.log 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.log: gzip/var/log/ceph/ceph-client.admin.44142.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49798.log 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44142.log.gz 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58393.log.gz 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43218.log 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56468.log 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49798.log.gz 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43218.log.gz 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42328.log 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39922.log 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56468.log.gz 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42328.log.gz 2024-09-12T07:23:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50404.log 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38834.log 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr: 92.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39922.log.gz 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50404.log.gz 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63396.log 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38834.log.gz 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43156.log 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.63396.log: -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.42069.log 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63396.log.gz 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43156.log.gz 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44665.log 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42069.log: -- /var/log/ceph/ceph-client.admin.45030.log 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42069.log.gz 2024-09-12T07:23:08.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44665.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44665.log.gz 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61727.log 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45030.log.gz 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51870.log 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61727.log.gz 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39163.log 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51870.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51870.log.gz 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39160.log 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39163.log.gz 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59591.log 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39160.log.gz 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45977.log 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.59591.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59591.log.gz 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62402.log 2024-09-12T07:23:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45977.log.gz 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44509.log 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62402.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62402.log.gz 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39540.log 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44509.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44509.log.gz 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39921.log 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39540.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39540.log.gz 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33739.log 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63522.log 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39921.log: gzip/var/log/ceph/ceph-client.admin.33739.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50326.log 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39921.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33739.log.gz 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51562.log 2024-09-12T07:23:08.518 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63522.log.gz 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45397.log 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50326.log.gz 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51562.log.gz 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49050.log 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45397.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45397.log.gz 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61002.log 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49050.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49050.log.gz 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38475.log 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61002.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61002.log.gz 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51582.log 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38475.log.gz 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54070.log 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51582.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51582.log.gz 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.39169.log 2024-09-12T07:23:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54070.log.gz 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44707.log 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38832.log 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39169.log.gz 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58178.log 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44707.log.gz 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46946.log 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38832.log.gz 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49840.log 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46946.log: /var/log/ceph/ceph-client.admin.58178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46946.log.gz 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61730.log 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58178.log.gz 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49840.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49840.log.gz 2024-09-12T07:23:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44200.log 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40080.log 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61730.log: /var/log/ceph/ceph-client.admin.44200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61730.log.gz 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62640.log 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44200.log.gz 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46906.log 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40080.log: /var/log/ceph/ceph-client.admin.62640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33336.log 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62640.log.gz 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40080.log.gz 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46906.log.gz 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62598.log 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33336.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.60787.log 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33336.log.gz 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53248.log 2024-09-12T07:23:08.521 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62598.log.gz 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.60787.log: -- /var/log/ceph/ceph-client.admin.62850.log 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60787.log.gz 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53248.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53248.log.gz 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50964.log 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62850.log.gz 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50185.log 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50964.log.gz 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49333.log 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50185.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50185.log.gz 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63480.log 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49333.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45737.log 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49333.log.gz 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51827.log 2024-09-12T07:23:08.522 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63480.log.gz 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5/var/log/ceph/ceph-client.admin.45737.log: --verbose -- /var/log/ceph/ceph-client.admin.60641.log 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45737.log.gz 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51827.log: -- /var/log/ceph/ceph-client.admin.43321.log 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51827.log.gz 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.60641.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60641.log.gz 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39030.log 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43321.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43321.log.gz 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45354.log 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39030.log.gz 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40406.log 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45354.log.gz 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63186.log 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41291.log 2024-09-12T07:23:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.40406.log: /var/log/ceph/ceph-client.admin.63186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39845.log 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63186.log.gz 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61729.log 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr: 95.6%/var/log/ceph/ceph-client.admin.39845.log: -- replaced with /var/log/ceph/ceph-client.0.40406.log.gz 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41291.log.gz 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39845.log.gz 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43097.log 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61729.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61729.log.gz 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40290.log 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43097.log.gz 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55621.log 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40290.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40290.log.gz 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61044.log 2024-09-12T07:23:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55621.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55621.log.gz 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39532.log 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61044.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61044.log.gz 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41629.log 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39532.log.gz 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61254.log 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41629.log.gz 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63438.log 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61254.log.gz 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47206.log 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.63438.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63438.log.gz 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53107.log 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47206.log.gz 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62892.log 2024-09-12T07:23:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53107.log.gz 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46299.log 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62892.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62892.log.gz 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54765.log 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46299.log: -- /var/log/ceph/ceph-client.admin.46058.log 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46299.log.gz 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54765.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54765.log.gz 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63270.log 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43663.log 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46058.log: /var/log/ceph/ceph-client.admin.63270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58590.log 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63270.log.gz 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46058.log.gz 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43663.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-12T07:23:08.526 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43663.log.gz 2024-09-12T07:23:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.58590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39782.log 2024-09-12T07:23:08.527 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58590.log.gz 2024-09-12T07:23:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-osd.3.log: -- /var/log/ceph/objectstore_tool.56217.log 2024-09-12T07:23:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39782.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39782.log.gz 2024-09-12T07:23:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.55924.log 2024-09-12T07:23:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44788.log 2024-09-12T07:23:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/objectstore_tool.55924.log: /var/log/ceph/objectstore_tool.56217.log: 94.5% -- replaced with /var/log/ceph/objectstore_tool.55924.log.gz 2024-09-12T07:23:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39386.log 2024-09-12T07:23:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44788.log.gz 2024-09-12T07:23:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61728.log 2024-09-12T07:23:08.527 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39386.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39386.log.gz 2024-09-12T07:23:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49599.log 2024-09-12T07:23:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.61728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61728.log.gz 2024-09-12T07:23:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62062.log 2024-09-12T07:23:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49599.log.gz 2024-09-12T07:23:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41849.log 2024-09-12T07:23:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.62062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62062.log.gz 2024-09-12T07:23:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41849.log.gz 2024-09-12T07:23:08.528 INFO:teuthology.orchestra.run.smithi046.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.56217.log.gz 2024-09-12T07:23:08.552 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-12T07:23:08.604 INFO:teuthology.orchestra.run.smithi064.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-12T07:23:08.781 INFO:teuthology.orchestra.run.smithi046.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-12T07:23:09.602 INFO:teuthology.orchestra.run.smithi046.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-12T07:23:09.688 INFO:teuthology.orchestra.run.smithi064.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-12T07:23:10.164 INFO:teuthology.orchestra.run.smithi046.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-12T07:23:10.666 INFO:teuthology.orchestra.run.smithi064.stderr: 96.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-12T07:23:12.095 INFO:teuthology.orchestra.run.smithi064.stderr: 96.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-12T07:23:12.202 INFO:teuthology.orchestra.run.smithi046.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-12T07:23:13.698 INFO:teuthology.orchestra.run.smithi064.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-12T07:23:13.700 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-09-12T07:23:13.700 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m5.329s 2024-09-12T07:23:13.700 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m12.410s 2024-09-12T07:23:13.700 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m0.488s 2024-09-12T07:23:15.445 INFO:teuthology.orchestra.run.smithi046.stderr: 96.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-12T07:23:15.448 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-12T07:23:15.448 INFO:teuthology.orchestra.run.smithi046.stderr:real 0m7.109s 2024-09-12T07:23:15.448 INFO:teuthology.orchestra.run.smithi046.stderr:user 0m13.951s 2024-09-12T07:23:15.448 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m0.683s 2024-09-12T07:23:15.448 INFO:tasks.ceph:Archiving logs... 2024-09-12T07:23:15.449 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413/remote/smithi046/log 2024-09-12T07:23:15.450 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T07:23:17.284 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413/remote/smithi064/log 2024-09-12T07:23:17.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T07:23:18.663 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T07:23:18.673 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T07:23:18.673 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T07:23:18.676 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T07:23:18.778 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T07:23:18.787 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T07:23:18.787 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T07:23:18.790 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T07:23:18.805 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-09-12T07:23:18.807 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-09-12T07:23:18.811 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T07:23:18.811 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-09-12T07:23:18.811 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +359us[ +328us] +/- 55ms 2024-09-12T07:23:18.811 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 41 -278us[ -310us] +/- 55ms 2024-09-12T07:23:18.811 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 -175us[ -175us] +/- 69ms 2024-09-12T07:23:18.811 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T07:23:18.812 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T07:23:18.813 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-09-12T07:23:18.813 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 +347us[ +357us] +/- 55ms 2024-09-12T07:23:18.813 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 -298us[ -298us] +/- 55ms 2024-09-12T07:23:18.813 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -185us[ -185us] +/- 69ms 2024-09-12T07:23:18.813 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T07:23:18.813 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T07:23:18.821 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T07:23:18.822 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T07:23:18.853 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T07:23:18.941 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T07:23:19.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-12T07:23:19.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T07:23:19.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-12T07:23:19.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T07:23:19.083 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T07:23:19.091 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T07:23:19.135 INFO:teuthology.task.internal:Duration was 673.070414 seconds 2024-09-12T07:23:19.136 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T07:23:19.178 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T07:23:19.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T07:23:19.181 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T07:23:19.224 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T07:23:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T07:23:19.332 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T07:23:19.333 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T07:23:19.333 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T07:23:19.354 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T07:23:19.354 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T07:23:19.382 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T07:23:19.382 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T07:23:19.397 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T07:23:19.450 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T07:23:19.450 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T07:23:19.466 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T07:23:19.610 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T07:23:19.620 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T07:23:19.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T07:23:19.646 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T07:23:19.672 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T07:23:19.686 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T07:23:19.689 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T07:23:19.710 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-09-12T07:23:19.736 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-09-12T07:23:19.781 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T07:23:19.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T07:23:19.796 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T07:23:19.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T07:23:19.837 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T07:23:19.846 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T07:23:19.847 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413/remote/smithi046 2024-09-12T07:23:19.847 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T07:23:19.889 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902413/remote/smithi064 2024-09-12T07:23:19.889 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T07:23:19.925 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T07:23:19.925 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T07:23:19.927 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T07:23:19.976 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T07:23:19.988 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T07:23:19.988 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T07:23:20.014 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T07:23:20.014 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T07:23:20.017 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T07:23:20.030 INFO:teuthology.orchestra.run.smithi046.stdout: 265325 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 07:23 /home/ubuntu/cephtest 2024-09-12T07:23:20.032 INFO:teuthology.orchestra.run.smithi064.stdout: 265895 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 07:23 /home/ubuntu/cephtest 2024-09-12T07:23:20.033 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T07:23:20.059 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T07:23:20.138 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} duration: 673.0704140663147 flavor: default owner: scheduled_dis@teuthology success: true 2024-09-12T07:23:20.139 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T07:23:20.226 INFO:teuthology.run:pass